Weather Data users typically have a varied systems, datasets and techniques they can work with to download the weather data they need. One common request is where they need extremely large datasets that would either take too long to generate, is unwieldy to download/transfer or it is just tedious to do daily, hourly or in realtime. This is where bulk datasets excel. Whether you want to prime a historical weather database for past analysis or you are in need of large quantities of locations, a bulk dataset can help greatly.
What types of queries are offered in bulk?
Visual Crossing offers two varieties of bulk downloads: Standard and Custom. Standard bulk datasets include queries that use typical geographic location sets for typical time frames of weather run on a consistent frequency. One example of this would be “US Zip Codes 15-day Forecast”. This data set is created on a daily basis for all US Zip Codes and includes all weather variables for a full 15-day forecast. Some of the bulk datasets are at the daily level, some at the hourly level and some will show real-time current conditions. But these datasets represent very common mass queries of weather data for common locations such as Zip Code, City or Weather Station.
Custom bulk queries are for customers who require specific locations, such as their store locations, or specific time periods or levels. Once example of this would be “My Store Locations for the last 14 days of Temperature and Snowfall”. For these types of scenarios we use the customer requirements to set up a custom query that runs at the timeframes specified by the customer. The bulk query will be available for download from the interface for them after completion. The customer simply needs to download it from the website or request it from a common download link, either through the web or script/code.
Sign up
To sign up simply review our Additional Offerings document and click to purchase: Bulk weather data in Visual Crossing.
Who is eligible to use the bulk feature?
The Standard Bulk Datasets are available to anyone as a subscription. This will give those customers access to the entire list of standard bulk datasets available.
The Custom Bulk Datasets are specialized because they will use the same resources to produce the standard datasets but are not shared among multiple customers. As such, these custom queries are quoted based upon the size and resource usage for that query. These scheduled datasets start at $35/month for a subscription.
How do I decide between Standard and Custom?
There are two main areas to consider when determining which path to choose. Custom Bulk queries are always more expensive so the user will have to weigh the value of having the exact dataset they require.
Location Accuracy
Visual Crossing Weather is a highly accurate weather system that provides weather data interpolated from many weather stations to account for your exact location. For some customers, accuracy is 100% required, for others the accuracy of Zip Code or City is close enough. We have seen varying temperature differences of 5 degrees F or more just between the city level and a nearby zip code. Of course this depends upon your area as well as some areas have varying characteristics such as altitude that can vary temperature and precipitation. If your store addresses contain zip codes and the zip code level accuracy is good enough for your analysis, then Standard Datasets may be right for you.
Excess Data
The other reason to choose a custom data set is when you don’t have the capability to join and filter out additional data. Remember that the standard datasets will include all of the data you need and oftentimes volumes of data you do not. If you have stores in 3,000 zip codes your standard bulk zip data set will bring back more than 30,000.
I signed up for bulk data, how do I retrieve it?
After you have signed up for a bulk data subscription on your account, your login will be enabled with the feature. Simply login to the standard web interface at:
https://www.visualcrossing.com/weather-query-builder/
Now you can login to your account and you should see the following ‘Download’ and ‘My Datasets’ button at the upper right of the web interface.
Simply click on the My Datasets button and you will see all available bulk data sets on your account as see below:
By clicking on any of the subscribed datasets we can see all of the available versions that have been run for your dataset.
On your datasets page you will see a list of the datasets and the date/time they were generated. At the top of the list is a special link to refer to the latest version of the dataset. This will help script-based downloads to always get the latest dataset without the need to update a date string.
Also you will find links to both download the text file dataset in it entirety. This dataset is in CSV format which can be imported to Excel and you may optionally change the filename extension from txt to csv. There is also a checkbox option to zip the file before download as needed.
Final Thoughts
If you need any additional help or need to request additional bulk datasets please email us at support@visualcrossing.com