Yesterday
Top Secret/SCI
Mid Level Career (5+ yrs experience)
IT - Database
Remote/Hybrid•Arlington, VA (Off-Site/Hybrid)
Full Visibility is seeking a Senior Data Engineer for hybrid work in Arlington, VA. Candidate must possess an active Top Secret clearance.
The Senior Data Engineer will design and optimizing data pipelines, leverage object storage solutions, and work with structured and unstructured datasets in a fast-paced environment. They will design, develop, and maintain scalable data pipelines for processing and analyzing structured and unstructured data. The candidate will develop and optimize data parsing and transformation processes. It is expected that the candidate will implement ETL workflows using open-source tools and frameworks. The candidate will identify and implement open-source solutions for efficient data handling and processing.
FV REQ 1600303
Requirements
● Active Top Secret Clearance
● At least five (5+) years of experience in data engineering, ETL development, or a related field
● Strong experience with object storage technologies (AWS S3, Azure Blob, MinIO, or similar)
● Proficiency in data parsing and transformation, handling structured and unstructured data
● Hands-on experience with ETL tools and data workflow orchestration (e.g., Apache Airflow, Luigi, Prefect)
● Strong programming skills in Python, SQL, or Scala
● Experience with open-source data processing tools (e.g., Kafka, Spark, Flink, Hadoop)
● Familiarity with database technologies (PostgreSQL, MySQL, or NoSQL solutions)
● Ability to work in a fast-paced environment with large-scale datasets
Preferred:
• Experience with forensic data processing or handling sensitive data
• Knowledge of data security and compliance best practices
• Experience with distributed computing frameworks and high-performance data processing
• Familiarity with containerization (Docker, Kubernetes) and cloud-based data engineering solutions
The Senior Data Engineer will design and optimizing data pipelines, leverage object storage solutions, and work with structured and unstructured datasets in a fast-paced environment. They will design, develop, and maintain scalable data pipelines for processing and analyzing structured and unstructured data. The candidate will develop and optimize data parsing and transformation processes. It is expected that the candidate will implement ETL workflows using open-source tools and frameworks. The candidate will identify and implement open-source solutions for efficient data handling and processing.
FV REQ 1600303
Requirements
● Active Top Secret Clearance
● At least five (5+) years of experience in data engineering, ETL development, or a related field
● Strong experience with object storage technologies (AWS S3, Azure Blob, MinIO, or similar)
● Proficiency in data parsing and transformation, handling structured and unstructured data
● Hands-on experience with ETL tools and data workflow orchestration (e.g., Apache Airflow, Luigi, Prefect)
● Strong programming skills in Python, SQL, or Scala
● Experience with open-source data processing tools (e.g., Kafka, Spark, Flink, Hadoop)
● Familiarity with database technologies (PostgreSQL, MySQL, or NoSQL solutions)
● Ability to work in a fast-paced environment with large-scale datasets
Preferred:
• Experience with forensic data processing or handling sensitive data
• Knowledge of data security and compliance best practices
• Experience with distributed computing frameworks and high-performance data processing
• Familiarity with containerization (Docker, Kubernetes) and cloud-based data engineering solutions
group id: 10230605
