Project case study
Maxwell — ML Data Pipelines
Data Engineering
GPU-accelerated pipelines processing 20+ years of market data with Dask, RAPIDS, and Apache Arrow
PythonPandasNumPyNumbaRAPIDSDaskApache ArrowApache ParquetScikit-learnNVIDIA CUDA
Custom libraries for acquiring, preparing, and storing financial market data. They provide WebSockets/REST connectors, anomaly detection and cleansing across hundreds of millions of data points, and domain-specific feature engineering to produce high-integrity modeling data.
Maxwell processed 20+ years of trading and order-book market data at multiple time resolutions from high-volume real-time market APIs, accelerating storage and retrieval with Apache Arrow in-memory columnar formats and Apache Parquet persistent on-disk storage.
Key capabilities:
- Automated labeling of time-series data with algorithmic trading attributes such as stop losses, long/short price targets, ATR, and VWAP
- Domain-driven feature engineering from technical indicators and statistical markers, with automated binning, one-hot encoding, scaling, and normalization tuned for market microstructure data
- ML pipeline orchestration tools managing data flow across modeling and algorithmic-trading workflows with fluid feature extraction
- GPU-accelerated (CUDA) processing with Dask and RAPIDS for hundreds of millions of data points