Senior Data Engineer

Are you ready to turn complex data into powerful mission insight? We're seeking a Senior Data Engineer at NS2 Mission with a passion for innovation and impact to join our team in Herndon and Chantilly, VA. In this role, you'll design and develop cutting-edge data pipelines, tools, and services that fuel national security and intelligence efforts. From structured logs to unstructured sensor feeds, you'll work across the data spectrum-enabling real-time analytics and supporting high-stakes decision-making.

This isn't just another data role-this is your chance to engineer solutions that truly matter.

Key Responsibilities:

• Design, develop, and optimize scalable data pipelines using technologies such as Apache Spark, Apache Iceberg, Trino, OpenSearch, AWS EMR, NiFi, and Kubernetes containers.
• Ingest and move structured and unstructured data using approved methods into enterprise or local storage systems.
• Create robust ETL processes and perform ongoing operations and maintenance (O&M), enhancements, and documentation aligned with best practices and coding standards.
• Develop tools and scripts to extract data from various repositories, perform required transformations, and format the data into reusable data models for exploration and analysis.
• Collaborate with data scientists and mission stakeholders to analyze source data formats and design appropriate transformations that meet analytical goals.
• Ensure end-to-end data quality, lineage, and security while maintaining metadata to track data provenance and access controls.
• Clean and preprocess data to prepare it for use in advanced analytics, machine learning models, and decision support systems.
• Work with enterprise data governance and working groups to enhance and align with organizational data standards and practices.
• Collaborate with software engineers to update, configure, and maintain integrated data services based on evolving requirements.
• Contribute to the development of documentation for each project, including ETL mappings, user guides, code access instructions, and operational procedures.

Required Skills and Experience:

• Proficiency in building and maintaining ETL pipelines using modern data engineering tools and frameworks.
• Strong experience working with cloud-based and containerized environments (e.g., AWS, Kubernetes).
• Deep understanding of working with diverse data types and formats, including structured, semi-structured, and unstructured data.
• Familiarity with data ingestion tools and platforms such as Apache NiFi, Spark, and related open-source technologies.
• Demonstrated ability to collaborate across teams, including data scientists, software engineers, data stewards, and mission partners.
• Knowledge of data governance principles, metadata management, and data quality best practices.
• Experience with implementing data privacy and access controls to ensure secure data usage.

Position Clearance Requirement:

TS/SCI with an FSP