ETL/Azure data engineer

Job title: ETL/Azure data engineer

Company: Cleo Consulting

Job description: Assignment: RQ07890 – Software Developer – SeniorJob Title: ETL/Azure data engineerRequisition (SS): RQ07890Start Date: 2024-10-21End Date: 2025-03-31Client: Justice Technology ServicesOffice Location: Onsite / 595 Bay Street, TorontoOrganization: Justice Technology ServicesMinistry: Ministry of Solicitor General# Business Days: 186.00NOTE: This position is currently listed as “Onsite”; however, the Assignment under this request will provisionally be “Hybrid”, working 7.25 hours per calendar day, sometime between 8:00-5:00 PM (excluding lunch breaks) Monday to Friday inclusive for three (3) days onsite and two (2) days remote (offsite), unless otherwise identified. These conditions are subject to change as the OPS reflects on its current situation. During the duration of the assignment, you will be under the discretion of the Hiring Manager’s requirements for the assigned Project.Must have’s:

• Power BI: Proficiency in Power BI, including data modeling, creating interactive reports and dashboards and utilizing advanced features.
• ETL/Microsoft Azure Synapse Analytics experience, including its architecture, data storage options, data ingestion mechanisms and security features.
• Dynamics 365 CE and/or F&O report development experience.

Nice to have:

• Azure cloud certifications (e.g. Azure fundamentals, Azure Power BI Data Analyst Associate)
• Dynamics 365 certifications

DescriptionSCOPE:

• The OPGT is developing “PGTIMS”, a new Dynamics CE based solution that will modernize OPGT’s legacy applications and systems. This project requires migrating historical data from the legacy solutions to a new data warehouse/lakehouse, building all the new data assets in the lakehouse and creating new PowerBI reports.

DELIVERABLES:

• As a member of the reporting team, you will be responsible deliver the MVP for reporting, build the new data warehouse/lakehouse and deliver the required reporting. A high-level list of deliverables for the data migration team follows:
• Data Warehouse Design: analyze business requirements and design an efficient data warehouse/lakehouse/golden layer on Microsoft Azure
• Data Integration and ETL: help develop and automate ETL processes to load data from Dynamics 365 CE and F&O into the new data warehouse/lakehouse (bronze/silver/golden layers). Help integrate data from diverse sources into the new data warehouse.
• Data Modeling: create logical and physical data models that align with the project’s reporting and analytical needs.
• Data Security and Governance: implement appropriate security controls to protect sensitive data within the data warehouse and reports.
• Reporting and Visualization: develop reports, dashboards and visualizations as per reporting MVP. Ensure the quality, accuracy and integrity of reports.
• Performance Optimization: fine-tune the data warehouse and reporting processes to optimize query performance and overall system efficiency.
• Other duties as assigned.

The Vendor’s Personnel will also be required to:

• Complete work and achieve milestones within the assigned deadlines.
• Notify the Cluster/Ministry project Manager in writing of any issues or other material concerns related to the Assignment Deliverables, as soon as he/she becomes aware of them
• Submit Deliverables for the Cluster/Ministry approval as they are completed.
• Comply with the Ontario Government and the Cluster/Ministry security procedures and practices
• Comply with the Ontario Government and the Cluster/Ministry architecture/technology standards and best practices
• Comply with the Ontario Government and the Cluster/Ministry Conflict of Interest and Confidentiality Guidelines
• Provide knowledge and skill transfer to a designated Cluster/Ministry staff; and
• Comply with the Ontario Government I&IT Directive, Operational Policy on the I&IT Project Gateway Process, and other applicable Guidelines, Standards and Procedures.

Experience and Skill Set Requirements

• The VOR Master Service Agreement which expires on April 5, 2025, leaves some Contracts with funding unassigned for fiscal 2025-26. If the current statement of work expires on March 31, 2025, the remaining funds can be used to exercise an option to extend the SOW beyond March 31, 2025, based on business case approvals. Such extensions will be allowable, only if the Master Service Agreement is extended beyond April 5, 2025, and be upon the same terms, conditions, and covenants contained in the SOW.
• The start date is subject to change based on security clearances and contract signing timelines.

SKILLS AND KNOWLEDGE:

• Data Warehouse/Lakehouse Concepts: Strong understanding of data warehousing principles, including data modeling, ETL processes, data integration, and data governance.
• Microsoft Azure Synapse: In-depth knowledge and experience with Azure Synapse Analytics, including its architecture, data storage options (e.g., SQL Data Warehouse, Data Lake Storage), data ingestion mechanisms (e.g., Azure Data Factory), and security features.
• Power BI: Proficiency in Power BI, including data modeling, creating interactive reports and dashboards, configuring data gateways, and utilizing advanced features such as DAX and Power Query.
• SQL and Query Optimization: Strong SQL skills for writing complex queries, optimizing database performance, and troubleshooting query performance issues. Understanding of indexing, partitioning, and query execution plans.
• Development experience with Dynamics 365 and understanding of the Common Data Model for Dynamics.
• Report development experience for Microsoft Dynamics 365 Customer Engagement (CE) and/or Finance and Operations (F&O)
• Familiarity with version control systems (e.g., Azure Repos) and collaboration tools (e.g., Azure DevOps) for managing code, tracking changes, and collaborating with team members.

CRITERIA:Criteria 1 – PowerBI (25 Points)

• Demonstrated experience working Power BI, creating interactive reports and dashboards, configuring data gateways, utilizing advanced features such as DAX and Power Query.
• Experience with SSIS, SSRS

Criteria 2 – ETL / Azure Platform (25 Points)

• Experience with Azure Synapse Workspaces and/or Azure Data Factory
• Azure cloud certifications

Criteria 3 – Dynamics 365 (25 Points)

• Demonstrated software development experience with PowerApps platform, Power Automate, Dynamics CE &F&O and producing reports from Dynamics

Criteria 4 – SQL / DB (15 Points)

• Demonstrated experience with data manipulation languages (T-SQL, PL/SQL), data definition languages, query performance analysis & tuning
• Demonstrated experience with SQL Server, Oracle, Azure SQL Databases

Criteria 5 – Azure DevOps (10 Points)

• Demonstrated experience with Azure DevOps for version control, managing code, tracking changes, and collaborating with team members.

Expected salary:

Location: Toronto, ON

Job date: Wed, 11 Sep 2024 22:19:45 GMT

Apply for the job now!