Software Engineer- Data Platform
Company: Twitter | Location: Bangalore | Exp: 4+ Years
Description:
As engineers on the Data Platform team, our mission is to build the fastest, most reliable, and largest-scale data processing technologies in the world – able to cope with ever-increasing volumes of data in real-time – and then apply them to the company’s most critical and fundamental data problems.
As a member of the team, you will be the “source of truth” for Twitter’s most fundamental data – such as Tweet content, engagement data, and user relationships – along with core metrics such as daily and monthly active users. You will surface these datasets in real-time to mission-critical products and business applications throughout the company. You will empower dozens of engineering teams, hundreds of co-workers, and millions of users to dream of new insights and new possibilities.
What You’ll Do:
If this sounds like a team you want to be a part of, fantastic! We are looking for experienced engineers who love writing code, data engineering, understanding our customers, and collaborating with teammates to ship useful software.
Sample projects we’ve built:
Real-time aggregations of interactions on tweets at ~5M/sec scale Unify our batch processing pipelines that count and validate user activity Use hidden Markov modeling to categorize users’ tweeting states
Who You Are:
You take satisfaction in building resilient, performant, and thoroughly tested distributed systems that can power the most business-critical applications. You get stuff done and thrive in a small group environment. You have a strong sense of ownership and a curiosity to understand how things work, even if they take you outside your area of expertise. You welcome feedback on are constantly looking for ways to improve yourself.
Qualifications:
On our team, we need people who:
Have 4+ years of professional experience
Have backend development experience with a solid foundation in data pipelines, distributed systems, large-scale data processing
Have proficiency with Scala, Java, C/C++ or Python
Show deep understanding in at least one data processing framework including Hadoop, Spark, Flink, KafkaStreams, or Dataflow.
Enjoy working with our internal customers and having empathy for their problems.
Embrace a growth mindset and want to improve ourselves, the team, our processes, and the products we work on.
Additionally, it would be nice if you had:
Success in developing in a hybrid-cloud environment
Experience with Lambda architectures and different ways of implementing them.
Working knowledge of ETL and a query language
Experience with on-call responsibilities
Experience needed:
4+ Years
Location:
Bangalore
