Introducing
Mar 6, 2024
Starting today, daily historical Flat Files are now included at no extra charge for all users, whether you're on a personal or an enterprise Polygon paid plan. This means that you can easily explore and download historical market data included in your subscription. Choose between our user-friendly online File Browser for direct downloads and S3 access for seamless integration into automated workflows. All data is available in a straightforward CSV format, tailored to save you time and simplify your workflow.
Quinton Pike, Polygon's founder and CEO, said, “We saw a lot of friction with users trying to systematically download historical data, and we felt like we could improve that experience. Having access to the right tool for the right task saves a lot of work, and we feel like S3 is a better tool for what those users are trying to accomplish.”
We see this as a pioneering step that sets a new benchmark in accessibility and user-friendliness, making high-quality, institutional-grade market data available to everyone, at an affordable price.
Since all Polygon paid plans now include Flat Files, getting started is easy since we are providing direct access to historical market data in a downloadable format. This includes daily updates to Trades, Quotes, and Aggregates, for Stocks, Options, Indices, Forex, and Crypto, consistent with your subscription level. Here’s how you can easily explore and download your data.
Our Web-based File Browser is by far the easiest way to access historical market data. You can start browsing without logging in, and no additional tools or downloads are required.
This tool is designed to streamline your access to data, offering a straightforward and efficient browsing experience, with secure downloading for logged-in users.
S3 access offers flexibility for interactive use, scripting automated jobs, and embedding S3 logic into your software for easy data management. It's compatible with a variety of clients, including the AWS S3 CLI, MinIO, Rclone, and the Python boto3 SDK.
For detailed configuration instructions, please visit our knowledge base article.
For the following examples, we will use the MinIO client. Before configuring your S3 client, you will need to obtain the S3 endpoint, bucket name, and your Access and Secret keys from the Polygon Dashboard. Once you have installed the client and have your credentials, run the following command to create your
mc alias set s3polygon https://files.polygon.io Your-AccessKey-Here Your-SecretKey-Here
Now that you have the client configured, let’s explore the top-level buckets or directories to see what is available. Listing operations work for anyone, even without specific subscription access to the Flat Files products, but to actually download a dataset you will need an active subscription.
$ mc ls s3polygon/flatfiles/ [2024-03-05 08:58:06 PST] 0B global_crypto/ [2024-03-05 08:58:06 PST] 0B global_forex/ [2024-03-05 08:58:06 PST] 0B us_indices/ [2024-03-05 08:58:06 PST] 0B us_options_opra/ [2024-03-05 08:58:06 PST] 0B us_stocks_sip/
For instance, if you want to look at the stock aggregates directory, you can list the files in that specific path as shown below:
$ mc ls s3polygon/flatfiles/us_stocks_sip/day_aggs_v1/2024/03/ [2024-03-01 19:00:19 PST] 200KiB STANDARD 2024-03-01.csv.gz [2024-03-04 19:00:21 PST] 200KiB STANDARD 2024-03-04.csv.gz
You can preview the contents of a file without downloading it which can be very useful to learn the format of these files and what to expect. This command displays the beginning of a file, providing a quick look at the data structure and content, which is especially useful for very large files:
$ mc cat s3polygon/flatfiles/us_stocks_sip/day_aggs_v1/2024/03/2024-03-04.csv.gz | gzcat | head -4 ticker,volume,open,close,high,low,window_start,transactions A,2169240,139.15,142.86,143.49,138.81,1709528400000000000,36438 AA,3954744,27.5,27.48,27.55,27.04,1709528400000000000,36282 AAA,37833,25.16,25.06,25.16,25.0101,1709528400000000000,66
If you decide to use a file for your analysis, simply download it directly to your local machine using the following command:
$ mc cp s3polygon/flatfiles/us_stocks_sip/day_aggs_v1/2024/03/2024-03-04.csv.gz .
With just a few easy commands shown in these examples, you are all set to leverage these extensive datasets to their full potential.
We're excited to roll this out for all our users, we firmly believe this represents a significant leap forward in making market data more accessible and manageable. The introduction of the web-based File Browser, in particular, highlights our commitment to delivering exceptional user experiences, offering an intuitive interface for data exploration and download.
You already have access to Flat Files, so we encourage you to start leveraging them right away. Happy exploring!
Team Polygon
editor
See what's happening at polygon.io
We are excited to announce our integration with QuantConnect! This offering empowers users with state-of-the-art research, backtesting, parameter optimization, and live trading capabilities, all fueled by the robust market data APIs and WebSocket Streams of Polygon.io.
jack
We are thrilled to announce our latest partnership with everviz, bringing embeddable visualizations of Polygon.io data to any platform.
jack
Polygon now includes daily historical Flat Files in all paid plans at no extra charge, featuring a new web-based File Browser and S3 access for simplified data exploration and integration.
editor