For some time, our Stock API has provided historical market data through a standard REST API. This is great for applications that need to request specific data points on demand. We are now introducing a new way to get this same data, designed for a different type of work: Flat Files S3 API.
This new product provides direct access to our complete historical market data archive as simple, downloadable files.
What is Flat Files S3 API?
It is an API that behaves like the Amazon S3 API. It allows you to list and download files from our data archive using the wide ecosystem of S3-compatible tools and software development kits (SDKs). Instead of making individual HTTP requests for price data, you can now pull entire datasets for any period, directly into your own environment.
The data is provided as GZIP compressed CSV files. This format is simple to work with in almost any programming language or data analysis tool. The files are organized in a logical path structure by exchange, data type, and date, making it straightforward to locate the exact information you need.
Key Features
- S3-Compatible Access: Use your existing S3 tools, like the AWS CLI or SDKs for Python (Boto3), Java, Go, and others. There is no new proprietary system to learn.
- Bulk Data Retrieval: Get direct access to our multi-petabyte archive of historical market data, including OHLCV, trades, and quotes.
- Simple File Format: All data is in compressed CSV, a universal format for data interchange.
- Efficient Filtering: Use path prefixes to list only the specific datasets you require, without retrieving large directory listings.
Who Will Benefit from Flat Files?
Flat Files API is built for users who work with large volumes of data. If your work requires analyzing months or years of historical information at once, this product is for you.
- Quantitative Analysts: Your backtesting models require deep historical data. Flat Files API lets you download years of OHLCV or trade data to test strategies against a wide range of market conditions.
- Data Scientists & ML Engineers: Training predictive models requires massive amounts of input data. You can pull extensive, clean datasets to feed directly into your machine learning pipelines.
- Academic Researchers: Your work often involves large-scale empirical studies. Now you can get the comprehensive datasets needed for financial or economic research without the typical access or formatting issues.
Example Use Cases
- Comprehensive Strategy Backtesting: A quant firm can download 20 years of daily OHLCV data for all tickers on a specific exchange to test a new long-term investment strategy, including delisted stocks to avoid survivorship bias.
- Training a Volatility Prediction Model: A machine learning engineer could pull five years of tick-level trade data for a set of securities to train a model that predicts short-term price volatility.
- Market Microstructure Analysis: A university researcher studying liquidity could download all quote data for a specific stock over a six-month period to analyze bid-ask spreads and order book depth around major news events.
- Populating a Data Warehouse: A company can use the API to perform an initial bulk ingestion of historical data into their own database, like Snowflake or BigQuery, for internal analytics.
You can get started by checking out the documentation for Flat Files S3 API today and downloading a free API key to make your first requests.
