Comcast shapes the future at the intersection of media and technology. Comcast provides one of the kind entertainment platform, High Speed Data platform, and Internet of Things services including Home Security services. We create world-class experiences that people love and trust and drive innovation that builds value. We bring millions TV and entertainment, sports and news, communications and home management, theme parks, home security, Voice, and high speed Internet access. Comcast brings to life the best of what’s to come. Comcast brings Entertainment, Internet, Voice, and other services to the customers via the largest footprint of networks in the country. 20M+ On-Demand video are streamed daily, 8200 Video and Audio Channels are streamed daily across the country, 167M+ phone calls are made and 153M messages are send daily on our network. Comcast supports more than 800K route miles or network pipeline.

As the Senior Big Data Engineer, you will work on the collecting, storing, processing, and analyzing Comcast’s Exabyte scale data sets; captured from tens of millions of devices. Your primary focus will be on choosing or building optimal solutions to use for these purposes; then maintaining, implementing, and monitoring them. This person will also be responsible for integrating them with the architecture used across the company.The Sr. Big Data Engineer will design and build large-scale data processing systems, be an expert in data warehousing solutions and should be able toworkwith the latest (NoSQL) database technologies. You will collect and document requirements to define a competitive system based on the needs of the parent organization. You will work with a team of architects and other engineers to create an incremental product design that implements the defined system. You will work with other engineers to implement that incremental design and deploy upgrades continually to production.

This is a unique opportunity for the right candidate to champion exciting new technology in a Fortune 50 organization and establish a reference implementation that could be adopted across extensive technology portfolio.You will be promoting benefits of big data processing and analysis across the organization and will need to confidently engage and present to application developers, infrastructure managers and senior business sponsors alike.

Responsibilities

*Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities

*Implementing ETL process

*Monitoring performance and advising any necessary infrastructure changes

*Defining data retention policies

Skills and Qualifications

*Proficient understanding of distributed computing principles

*Develop automation and management capabilities of Hadoop cluster, with all included services

*Ability to troubleshoot and solve any ongoing issues with operating the cluster

*Proficiency with Hadoop v2, MapReduce, HDFS

*Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming

*Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala

*Experience with Spark

*Experience with integration of data from multiple data sources

*Experience with NoSQL databases, such as HBase, Cassandra, MongoDB

*Knowledge of various ETL techniques and frameworks, such as Flume

*Experience with various messaging systems, such as Kafka or RabbitMQ

*Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O

*Good understanding of Lambda Architecture, along with its advantages and drawbacks

*Experience with Cloudera/MapR/Hortonworks

*Knowledge of CEPH filesystem

*Demonstrated ability to conceive, manage, and complete software deliverables

*Linux systems administration skills, across distributions, and especially in a cloud or virtualized environment

*Understanding of IP networking and traffic scaling

*Experience with agile development methodologies, rapid application development, and project management

*Proven ability to design and present understandable and practical solutions to complex problems

*Demonstrated leadership skills in a fast-paced, team-driven environment

*Strong verbal and written communication skills, including visual presentation skills

*Demonstrated experience in research data collection, analysis, and presentation

*Experience with intellectual property portfolio management, especially patents and trademarks

*Ability to work effectively across internal and external organizations

*Ability to travel when needed; expected travel is 5-25%

*Ability to promote technologies to large audiences or top level executives

Comcast is an EOE/Veterans/Disabled/LGBT employer

Desired Skills and Experience