Big Data/Hadoop/Splunk Tester -- KUMDC
Job Description
Job Summary
We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines.
The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems.
You will design and implement automated data quality tests for ingestion and transformation layers (Bronze ? Silver ? Gold), enforce schema and data contracts, detect drift, validate feature pipelines, and monitor data anomalies.
Key Responsibilities
- Develop and execute automated data quality tests for ingestion and transformation layers.
- Enforce schema and data contracts across pipelines.
- Detect and address data drift and anomalies.
- Validate feature pipelines to ensure accuracy and reliability.
- Monitor data quality and maintain issue logs for critical pipelines.
- Deliver production-grade validation suites and provide QA sign-off for key data processes.
Required Qualifications
- 8+ years of experience in SQL.
- 5+ years of experience with Databricks and Spark.
- 3+ years of experience in data quality frameworks (e.g., Great Expectations, Deequ, Soda, dbt tests).
- 3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) for test generation and validation workflows.
- 5+ years of experience delivering production-grade validation suites and QA sign-off for critical pipelines.
How to Apply
Ready to start your career as a Big Data/Hadoop/Splunk Tester -- KUMDC at Compunnel Inc.?
- Click the "Apply Now" button below.
- Review the safety warning in the modal.
- You will be redirected to the employer's official portal to complete your application.
- Ensure your resume and cover letter are tailored to the job description using our AI tools.
Frequently Asked Questions
Who is hiring?▼
This role is with Compunnel Inc. in Halifax.
Is this a remote position?▼
This appears to be an on-site role in Halifax.
What is the hiring process?▼
After you click "Apply Now", you will be redirected to the employer's official site to submit your resume. You can typically expect to hear back within 1-2 weeks if shortlisted.
How can I improve my application?▼
Tailor your resume to the specific job description. You can use our free Resume Analyzer to see how well you match the requirements.
What skills are needed?▼
Refer to the "Job Description" section above for a detailed list of required and preferred qualifications.