I am what is called today a full-stack data professional.
I have years of experience working on data systems for financial services companies across many roles (titled or not): business/data analyst, analytics engineer, data engineer, full-stack software engineer/developer, data scientist — and in a few brief moments even DataOps/DevOps engineer and QA engineer. Basically I am always there to do what needs to be done for the company when duty calls.
...
(ranked by my proficiency in them)
- Python: my mother tongue; the lingua franca of the MAD (ML/AI/data) world.
- JavaScript/TypeScript: JS is a messy language with a sad type system, but it is unavoidable as it is the lingua franca of web-based front-end apps.
- PHP: 7.4 and 8.x are palatable when compared to 5.x, but... nope, I'm not working with PHP ever again
- good ol' Java
- good ol' C
- Scala 3: I will pick any time over Java for JVM; much more expressive, with a sound type system.
- Haskell: it has some flaws, but it is a pleasure to work with.
- Clojure
- Orchestration:
- Apache Airflow (on-premise)
- Amazon MWAA (managed Airflow on AWS)
- Google Cloud Composer (managed Airflow on GCP)
- Ingestion:
- (Py)Airbyte (on-premise)
- Amazon AppFlow (AWS)
- Google Cloud Data Fusion (GCP)
- Transformation:
- dbt Core (on-premise and AWS)
- BigQuery Dataform (GCP)
- Relational Database:
- PostgreSQL or MySQL (on-premise)
- Amazon Aurora (AWS)
- AlloyDB (GCP)
- Object Storage:
- S3-compatible MinIO (on-premise)
- Amazon S3 (AWS)
- Google Cloud Storage (GCP)
- Data Warehouse/Lakehouse:
- warehouse-like Citus or lakehouse-like Dremio/Trino (on-premise)
- Amazon Redshift (AWS)
- BigQuery (GCP)