Principal Engineer - Data Engineering
Bengaluru, Karnataka, India
Posted on Friday, January 19, 2024
Who is AiDash?
AiDash is making critical infrastructure industries climate-resilient and sustainable with satellites and AI. Using our full-stack SaaS solutions, customers in electric, gas, and water utilities, transportation, and construction are transforming asset inspection and maintenance - and complying with biodiversity net gain mandates and carbon capture goals. Our customers deliver ROI in their first year of deployment with reduced costs, improved reliability, and achieved sustainability goals. Learn more at www.aidash.com.
What you will do?
- Architect and implement scalable data processing platform for managing large-scale, heterogeneous geospatial data, including satellite imagery, LiDAR data, weather information, and enterprise datasets. Ensure these systems can efficiently ingest, store, and process diverse data types to support various analytical and operational needs.
- Implement rigorous data quality control measures and standardisation processes to maintain the integrity and accuracy of datasets. Develop protocols for data validation, cleaning, and deduplication, ensuring high-quality data is available for critical decision-making processes.
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and implement systems that enable data-driven decision-making.
- Stay abreast of industry trends and innovations in data engineering and propose the adoption of new technologies, tools, and methodologies to improve our data systems.
- Evolve AiDash architecture as we continue to scale. · Lead large initiatives across teams (refactor service to improve code quality, runtime efficiency, resource optimisation, keep stack updated, propose new technologies etc)Be in-house expert and provide consultation to product engineers to help them build scalable and reliable systems.
- Provide technical direction and leadership to drive large initiatives and execute key org goals.
What are we looking for?
- Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related field.
- Minimum of 12+ years of experience in data engineering with a proven track record of designing and operating large-scale data systems.
- Experience building and maintaining modern data pipelines in the cloud or hybrid.
- Strong experience with big data technologies (e.g., Hadoop, Spark), database systems (e.g., SQL, NoSQL), and ETL tools.
- Strong hands-on expertise with at least one programming language: Scala, Java, or Python.
- Deep understanding of data modelling, data warehousing solutions, and data architecture strategies for both transactional and analytical systems.
- Experience with cloud services (AWS, Azure, Google Cloud) and understanding of data pipeline tools and services offered by these platforms.
- Excellent problem-solving, analytical, communication & teamwork skills.
- Leadership experience and the ability to work cross-functionally with other teams to achieve company goals.
What other experience will make you a great candidate?
- Experience in building platform on geospatial data management and processing.
- Hands on experience in writing, designing image processing system, complex operations on geo data.