Ihre Aufgaben:
* Develop and maintain automated test scripts in Databricks (using Python, PySpark, or SQL)
* Validate JSON messages received via AMQP for structure, completeness, and accuracy
* Implement data quality checks (e.g., schema validation, range checks, anomaly detection)
* Ensure data integrity and reliability by designing and executing automated checks on incoming data streams and recording results in Azure DevOps
* Validate data flow from SCADA through AMQP to Databricks
* Integrate tests into CI/CD pipelines for continuous validation using Azure DevOps
Ihre Qualifikationen:
* Strong experience with Databricks, PySpark and JSON message handling
* Knowledge of Azure services (Data Lake, Blob Storage, Functions)
* Experience in setting up and running automated tests in Cloud Environments
* Proficiency in Python for scripting and automation
* Familiarity with CI/CD tools (Azure DevOps)
Ihre Vorteile:
* Start date: asap
* Remote-option