Description

The DarkStar Group is seeking a Platform and HPC Engineer with a TS/SCI + CI Poly clearance to join one of our top projects in Herndon, VA.  Below is an overview of the project, as well as information on our company, our benefits, and our $25,000 referral program.

THE PROJECT

The DarkStar Group's team solves unique and challenging intelligence problems for a Special Operations customer. This work is as close to the mission as a technologist can get, so the environment is fast-paced: team members face rapidly-changing requirements and priorities as mission needs evolve. If you hate monotony and want to use your skills to have a direct impact on real-world operational success, this is the project for you.

We are a multi-faceted software development and systems administration team working to build and maintain software applications backed by a self-managed cloud infrastructure (OpenStack) with a true big-data footprint (over 10 petabytes). Our diverse background of experience in mission support and software development serves as a catalyst to solve unique and challenging intelligence problems in support of special operations analysts and their on-going activities. Prototyping and frequent, iterative feedback are core to our delivery approach, anchored by a need to work quickly in support of our missions.

The technical stack is quite robust and includes Java, Python, C#, C/C++, Geospatial tools, Big Data and Graph Products (Hadoop, MapReduce, Spark, ElasticSearch, Neo4j), Linux, OpenStack, AWS, Ansible, SQL/NoSQL, Text Processing, Cloud Services, Containerization, Infrastructure as Code (IAC), and more.

Work on this program takes place in the Herndon, VA area (we cannot support remote work) and requires a TS clearance and a willingness to obtain a CI Poly: a current TS/SCI + CI Poly is preferred.

THE ROLE

• The DarkStar Group is seeking a skilled Platform and HPC Data Engineer to support the design, implementation, and optimization of data management solutions in high-performance computing (HPC) environments. The ideal candidate will have extensive experience working with various file systems, data labeling/tagging systems, and the configuration of a wide range of storage appliances. This role involves ensuring that data workflows, storage configurations, and metadata management are efficient, scalable, and aligned with organizational and government security requirements.
• The successful candidate will work within a cross-disciplinary team to support the technical needs of HPC platforms, data management, and large-scale computational workflows.

Key Responsibilities:

• Platform and HPC Data Engineering: Design and implement data management systems and architectures for HPC platforms, focusing on optimizing data flow, storage, and access in large-scale computing environments.
• File System Management: Oversee the configuration, maintenance, and optimization of distributed file systems (e.g., Lustre, IBM Spectrum Scale, NFS, GPFS) and storage solutions used in HPC environments to ensure efficient performance, scalability, and reliability.
• Data Labeling and Tagging: Implement and manage metadata-driven systems for data labeling/tagging. This includes the development of strategies for classifying, indexing, and organizing datasets to enhance data discoverability, access control, and auditing.
• Storage Appliance Configuration: Configure and maintain various storage appliances (e.g., NetApp, Dell EMC, HPE) and integrated storage solutions. Ensure that storage devices are optimized for performance, capacity, and availability within the HPC ecosystem.
• Data Integration and Workflow Optimization: Integrate data storage and management systems with HPC clusters, ensuring seamless data flow between compute nodes and storage appliances. Optimize data pipelines to support high-throughput workloads and minimize bottlenecks in I/O performance.
• Performance Tuning: Monitor and improve the performance of storage systems, focusing on I/O throughput, latency, and efficient resource allocation. Use performance metrics to guide optimizations across storage appliances and file systems.
• Security and Compliance: Implement security best practices for data access, protection, and management, ensuring compliance with government regulations and internal data governance policies. Configure encryption, access control, and secure data sharing methods.
• Automation and Scripting: Develop and maintain automation scripts (e.g., using Python, Bash, or Perl) to streamline storage configurations, data labeling/tagging, and system monitoring tasks. Automate processes related to data integration and HPC platform management.
• Collaboration and Support: Work closely with data scientists, HPC administrators, software developers, and other technical staff to support ongoing projects. Provide expertise in troubleshooting data storage issues and ensuring optimal system performance.
• Documentation and Reporting: Maintain thorough documentation for storage configurations, file system setups, data labeling/tagging procedures, and performance optimization strategies. Provide regular reports on system health, data management processes, and any improvements made.

Required Skills

• Education: Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field. A Master’s degree or higher is a plus.
• 7+ years of experience in managing data infrastructure in HPC environments, with expertise in file systems, storage appliances, and data workflows.
• Hands-on experience with distributed file systems, including Lustre, IBM Spectrum Scale (GPFS), NFS, and others commonly used in HPC settings.
• Proven experience with storage appliance configuration (e.g., NetApp, Dell EMC, HPE, or similar systems), including performance tuning, capacity management, and reliability.
• Strong experience in implementing data labeling/tagging systems, metadata management, and structuring large datasets for efficient access and compliance.
• Knowledge of high-performance networking protocols (e.g., InfiniBand, RDMA) and their role in data transfer and storage optimization.
• Familiarity with data access protocols like GridFTP, rsync, and NFS for large-scale data transfer.

Desired Skills (Optional)

• Experience with cloud storage integration or hybrid cloud environments, with knowledge of cloud-native storage solutions (e.g., AWS S3, Ceph, OpenShift).
• Familiarity with high-performance computing (HPC) schedulers (e.g., SLURM, PBS, Torque) and their interaction with data storage systems.
• Understanding of data protection mechanisms, including data replication, backup strategies, and disaster recovery in HPC environments.
• Experience with containerization (Docker, Singularity) in an HPC context for data processing and application deployment.
• Experience with machine learning or data science workflows in HPC environments.

About The DarkStar Group

Our Company

The DarkStar Group is a small business that solves BIG problems. We're one of the Inc. 5000 fastest-growing private companies in the US, and our engineers and scientists support the most critical national security missions in Virginia, Maryland, and elsewhere. Data Science, Software Engineering, Cloud/AWS Infrastructure, and Cyber/CNO are our core areas of expertise. We offer interesting and important work, job security, some of the best and most flexible benefits you'll find in the IC, and salaries so strong that they'll likely surprise you. 

Our Benefits

The DarkStar Group offers exceptional compensation and benefits:

• very strong salaries;
• 100% company-paid medical, dental, and vision premiums for you and all dependents;
• the ability to get increased salary if you don't need medical/dental/vision;
• 100% company-paid disability and life insurance benefits;
• a generously-funded HSA;
• an 8% 401(k) contribution; 
• 31 days of PTO/holidays to start (more with tenure);
• the ability to flex time across pay periods without using your PTO;
• a generous training budget;
• $25,000 employee referral bonuses;
• business development / growth incentives; and
• top notch company swag.

** We have a huge growth opportunity, so we are offering up to a $25,000 reward for anyone new you refer whom we hire. **

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.