Naineel Soyantar
Menu

Ahmedabad, India · Data Engineer I / II

Naineel Soyantar

I build data systems that survive the messy parts.

See selected workRead the field notes

Current frame

Software Developer 1 at ZURU Tech

I work across configuration-driven ingestion on AWS, Delta Lake pipelines, modular dbt projects, Terraform, and backend reliability.

Read the public resumeOpen GitHub

Selected work

Systems with evidence attached

Four projects carry different weight because their evidence differs.

01

IoT Lakehouse Pipeline

A production-inspired reference pipeline for streaming and batch IoT analytics.

Signal path05 stages
System contextPreview
IoT lakehouse pipeline architecture

Streaming telemetry reaches Delta Lake through Kafka and Spark. DuckDB reads the tables for data-quality reports.

Select a stage to keep its context. Press Escape to clear it.

Problem

Telemetry needs reliable ingestion, quarantine rules, repeatable writes, and useful analytics models.

Role

I built the end-to-end reference implementation.

System

  • Kafka ingestion
  • Spark Structured Streaming
  • Delta Lake Bronze, Silver, and Gold layers
  • Idempotent MERGE writes and quarantine checks
  • DuckDB data-quality reports

Evidence

Streaming and batch paths share one lakehouse model. Resume source

Silver processing validates and quarantines bad records. Resume source

Next run

Add repeatable failure drills and publish measured recovery behavior.

Open the repository

02

Spine

A configuration-driven Python framework for multi-source API ingestion on AWS.

Execution ledger05 stages
System contextPreview
Spine ingestion framework architecture

Spine validates one configuration shape, builds an execution plan, and applies shared extract and load strategies across sources.

Select a stage to keep its context. Press Escape to clear it.

Problem

One-off source integrations made onboarding and maintenance repeat the same work.

Role

My team built and open-sourced Spine. I contribute to the framework.

System

  • Python
  • AWS
  • Configuration-driven ingestion
  • Delta Lake

Evidence

Reduced onboarding time for new sources by 80%. Resume source

Next run

Document more source patterns and make operational recovery easier to test.

Open the repository

03

Kāinga

An AI real-estate recommendation product built for a Databricks hackathon.

Dark Kāinga chat interface with suburb comparison prompts, a diagram suggestion, and a question field.

Problem

Home discovery needed a conversational path across user needs and property data.

Role

I built the core chat application and user experience.

System

  • Next.js
  • FastAPI
  • Databricks Apps
  • Lakebase
  • Mosaic AI Agent Framework
  • Genie

Evidence

Placed 2nd among 39 APAC teams. Resume source

Won a $2,500 prize. Resume source

Received an opportunity to represent ZURU Tech at a future Databricks event. Resume source

Next run

Test the recommendation flow with more housing edge cases and document where the agent needs human review.

Open the repository

04

Cyclops

Open-source Kubernetes UI work across backend delivery, Helm, and observability.

Cyclops deployment view with three running pod rows and a View Logs control for each pod.

Problem

Operators needed clearer workload detail, controllable log streaming, and visible cache behavior.

Role

I contributed product and platform features in Go and Helm.

System

  • ReplicaSets UI for Deployments
  • SSE log streaming with abort control
  • Prometheus cache metrics for cyctl

Evidence

Implemented the ReplicaSets interface for Deployments. Resume source

Built SSE log streaming from backend to frontend with abort control. Resume source

Added cyctl cache metrics to Prometheus observability. Resume source

Next run

Keep the streaming path observable and cover disconnect behavior with focused integration tests.

Open the repository

Experience

One company, wider systems

2026–Present

Software Developer 1

ZURU Tech, Ahmedabad

  • Led a Terraform migration of core data services from ap-southeast-2 to us-east-1 with zero downtime and 99.99% availability. Resume source
  • Helped build and open-source Spine, reducing new-source onboarding time by 80%. Resume source
  • Moved fragmented Databricks and dbt scripts into a modular dbt project structure. Resume source
  • Set up AWS SSO access through Azure AD groups and Terraform-managed least privilege. Resume source

2024–2025

Junior Software Engineer

ZURU Tech, Ahmedabad

  • Built a FastAPI service over 1M+ SAP HANA SKU records with sub-second search. Resume source
  • Improved social-platform analytics ingestion throughput by 60% with asynchronous processing. Resume source
  • Designed Terraform modules for S3, EventBridge, Databricks catalogs, external locations, and Delta Sharing foundations. Resume source

Working principle

I keep starting again.

I have failed many times, and I still fail. I return with a smaller next step and a better question.

One line from Batman Begins stays with me:

Why do we fall, sir? So that we can learn to pick ourselves up.

Alfred Pennyworth, Batman Begins

Picking myself up means keeping the lesson, changing the next decision, and doing the work again.

Failure log

The useful parts of a bad run

I will publish a failed project here when I can connect the original decision to the break and the change that followed. Until then, this section remains incomplete.

First postmortem pending

  1. 01

    The assumption

  2. 02

    What broke

  3. 03

    What the evidence changed

  4. 04

    The next run

Operating notes

What survives another run

  1. 01

    I build side projects to learn systems end to end.

  2. 02

    I remove repeated development friction when a reusable tool can replace it.

  3. 03

    I contribute to open source and keep the repository as evidence.

  4. 04

    I test infrastructure ideas in reproducible repositories.

  5. 05

    I write postmortems, notes, and tutorials after the system has taught me something.

  6. 06

    Some projects ship. Some fail. I keep the useful parts and start another run.

Writing

Field notes from the work

Open all writing

27 December 2024 · 6 min read

A Date with Daytona: Exploring AstroJS and Sanity CMS

Recently, while working on my blog application, I discovered **Daytona**, a powerful development environment manager, through the ongoing Quest 23 (quira.sh). In this blog, we’ll explore how AstroJS, Sanity, and Daytona significantly enhanced the development process of this blog application.