Job Duties:
Develop, review and analyze complex, large-scale BigData/Cloud (AWS) solutions for tactical and strategic business objectives, that require in-depth evaluation of existing software applications. Build and maintain code to populate HDFS, Hadoop with log events from real-time data feeds or data loaded from SQL production systems. Responsible for assessing the impact of external production system changes to Big Data systems on Spark to implement changes to the ETL to ensure consistent and accurate data flows. Build hive scripts to ingest millions of records and responsible for performance tuning of these scripts. Use Sqoop to populate the data from RDBMS to HDFS and develop Hive/pig scripts to perform analysis. Design and develop UNIX shell scripts as part of the ETL process to automate the process of loading and pulling the data. Deliver advanced and complex reporting solutions including Dashboards, Standardized Reporting using Power BI. Perform Integration testing, production deployment through GitHub. Write stored procedures, packages, functions, and views in PL/SQL. Work in Weddington, NC and/or various client sites throughout the U.S. Must be willing to travel and/or relocate.