As a DevOps engineer in the engineering team, you will be responsible for ensuring that our data pipeline and platform run smoothly over time and that developent teams can quickly, easily and safely deliver and deploy their work by:
- Administering our AWS cloud infrastructure in a repeatable and automated way (Infrastructure as Code, terraform, ansible, ...)
- Supporting CI/CD workflows
- Monitoring our servers and services. (sentry, influx, grafana, ...)
- Automating (python, shell scripting, ...)
- Helping with containerization (docker) and orchestration (kubernetes) of agents and services
In order to be successful, you will need to bring your skills and knowledge around these technologies:
- AWS, and as you will be responsible for handling the infrastructure.
- Terraform, and ansible to manage infrastructure and configuration automatically in a controlled and repeatable way.
- Python, as it is the base of our tech stack at many levels.
- Linux shell command line, of course.
- PostgreSQL at an admin level (backups, monitoring).
- RabbitMQ, Celery
- Docker and Kubernetes, to deploy, orchestrate and scale services and agents.
- You need to be a developer, because you will need to write scripts to automate many tasks and understand development team needs.
You will learn on the job many of the other tech pieces we use. Of course it will be easier if you are already familiar with any them:
- PostgreSQL at an advanced admin level (advanced tuning, clusterization).
- InfluxDB, Grafana
Backend team is responsible for collecting massive amounts of data from fashion related websites and placing it into our analytics data repository. There is a whole data pipeline that starts with hundreds of spiders (python/scrapyd) that continuously crawl websites, continues with a series of data quality and data enrichment processes (python, celery, rabbitMQ, SQL...) and ends dumping clean, validated and normalized product information for our customers to analyze. As an example, the pipeline includes data enriching with machine learned models and massively parallel processing using Spark. The pipeline consists of many pieces that need to be monitored, scaled up and down, maintained, deployed, enhanced and adapted to new business needs continuously. The database already contains more than 500 millions of products (refreshed daily) and we see 1-2M of new products every week.
Our motto is "We love data". And we love technology that deals with data because it enables us to do incredible things... things that are valuable for our customers and that sustain a business.
StyleSage is (no longer) a startup founded 8 years ago with offices in New York and Madrid. Madrid is the home for our core technical team. It's an open, diverse and inclusive team of very skilled and talented individuals that are happy to collaborate, share knowledge and enjoy building great software together. We are looking forward to welcoming additional members for this team.
What We Offer
- First and foremost: permanent contract and competitive salary.
- Teams are made of people, not resources.
- Open, diverse and inclusive environment.
- A challenging and fun project to work and grow with, with the latest technologies, best practices and light speed evolution, all in a friendly, relaxed and positive environment.
- Fixed yearly training budget to spend on english classes, courses, books, or conferences.
- Your brand new laptop with OS of your choice (we recommend MacOSX or any flavor of Linux).
- A team of colleagues that will share a lot of knowledge with you (we have weekly in-depth internal talks).
- Fully remote position. Our office in Madrid is in a co-working space quite close to Avenida de América metro station, always available for you if you want to meet your colleagues in person and enjoy the fruit/coffee/tea we stock in there.