While many weather data projects are well served with hourly historical weather records, some have the need for historical weather reports as close as possible down to an exact minute. If your project needs this extra level of finely-detailed data, you already know that it can be difficult and expensive to obtain. Luckily, Visual Crossing Weather, the leader in high-quality, low-cost weather data now has released sub-hourly historical weather data that is cost-effective enough to be included in many more weather projects than previously possible. In this article, we’ll define exactly what you can get with sub-hourly historical weather data, what applications it is ideal for, and what limitations it has.
(Note: In this article we’ll refer to this extremely detailed weather data recorded at resolution chronological higher than hourly as sub-hourly data. However, you may also see it referred to as minute-wise data, minute-by-minute data, or to-the-minute data. These are all synonyms for the same type of weather data feed. So don’t be confused if you see these other references. Unfortunately, sub-hourly historical weather data is still uncommon enough that there isn’t yet an industry standard naming.)
Ready to retrieve sub-hourly data? Read the how to guide.
What is Sub-hourly Historical Weather Data?
In its most basic definition, sub-hourly historical weather data refers to weather data that is collected and reported in time increments less than one hour. However, that basic definition covers over the complexities of the weather reporting station network. While developers who use hourly weather data can typically remain blissfully unaware of these complexities, those who want to use chronologically higher resolution data need to have a more sophisticated understanding.
An Introduction to Weather Station Reporting
All weather data is reported with a specific timestamp and that timestamp is almost never exactly at the very beginning of an hour. This is true even for so-called hourly weather data. In fact, many official weather stations such as those at airports and military installations make their “hourly” report several minutes before each hour. It is quite common to see a station submitting their 6:00pm weather report several minutes early at 5:55pm or 5:57pm. This allows a global network of thousands of stations to not flood the central servers at exactly the same instant and also allows the central servers to produce the combined 6:00pm weather report as soon as possible after 6:00pm. Since those few minutes don’t typically matter in an application looking at hourly weather reports, the standard is to combine all of those “approximately” 6:00pm reports into a single, definitive 6:00pm observation.
A related fact is that is that many weather reporting stations, especially less-official ones such as municipal locations, schools, etc., make their weather reports in a repeating stream throughout the hours. Some report every 5 minutes, some every 10, and still others every 15 minutes. Also, these reports are not synced nicely with markings on a clock face or with each other. For example, one station may report at 6:04pm, 6:09pm, 6:14pm, 6:19pm, 6:24pm, and 6:29pm while another reports at 6:06pm, 6:21pm, and 6:36pm. Another unfortunate fact is that sometimes stations will skip reports or go offline entirely for random periods.
One final but important fact is that not all stations report all weather metrics. While weather metrics such as temperature and humidity are relatively easy to monitor and thus essentially universal, other metrics are not. Metrics such as cloud cover and snow fall require much more sophisticated monitoring hardware. So when you consider individual reports from individual stations, you will frequently find them missing some weather metrics. While this is a relatively minor problem when considering only official reporting stations, it can become an important problem if your project relies on more difficult-to-monitor metrics.
So when you want to use sub-hourly weather data, you need consider how all of these disparate and incomplete weather reports are harmonized into a useful whole.
Can You Simplify This for Me?
Fortunately, the answer is “yes!” Visual Crossing Weather has two options when retrieving sub-hourly historical weather reports. The simplest option is to let our weather engine take the raw weather reports and stitch them into a coherent weather picture for you. This is the optimal path when you need to get the most accurate weather conditions at a given location and at a specific time or every few minutes within a time range. There is also an option for more sophisticated users who want to consume the raw, timestamped weather records exactly as they are returned from reporting stations. This gives you the most detailed insight possible in exchange for your application assuming the burden of handling the additional complexity.
Simple Time Ranges
In the simplified approach to sub-hourly weather reports, the Visual Crossing Weather engine uses interpolation, geo-localization, and a complex set of rules to identify the best set of weather metrics for any given point at any given time. The engine can do this down to the minute by scanning reports from all stations in close proximity to the requested location. It takes into account both reporting times and proximity to calculate the best weather report for that location. It also automatically accounts for stations that are not reporting during a given period to help ensure that missing, incomplete, and potentially corrupt data doesn’t factor into a report.
The good news is that the data user can remain entirely unaware of the complex look-ups and calculations required to arrive at the final report. The user simply requests the specific time or the time range of interest. Using this option, the user can request the weather conditions at 1600 Pennsylvania Avenue on 1/1/2019 at 8:23am. Or, if the user would prefer the condition summaries over a time range, she could ask for the conditions at #10 Downing Street on 1/1/2020 between 8:00am and 10:00am in 10 minute increments. Even though we know that there is unlikely to be a station reporting at exactly the right minute to match any given query, the system will automatically evaluate the various weather reports available nearby both physically and chronologically to produce the best report for the exact location and time. In the first example, a single record will be returned. In the second example, 6 results will be returned for each hour, making 12 total. Each of these results will be for a 10-minute window, 8:00am-8:09am, 8:10am-8:19am, 8:20am-8:29am, and so on.
Using simple range queries, a sub-hourly weather data query is as simple to use as the more traditionally hourly or daily query type. The benefit is that you can get data at a much higher chronological resolution than previously possible.
Raw Weather Reports
Using raw weather reports gives access to the lowest level of weather data available in exchange for the data user taking on most of the responsibility for finding the best reports to match any specific use case. In this usage, the query look similar to the simple range query described above. The query gives a specific location and time range. However, the results are very different. Instead of getting a clean report or set of reports at standard time increments, the result contains a weather record for every nearby station that reported during the requested time.
As an example, let’s consider a query from 8:00am to 8:15am. There may be three nearby reporting stations, so for this example we’ll name them Station A, Station B, and Station C. You query results could look approximately like this. (Table simplified for illustration)
| Station | Station Details | Time | Temperature | Precipitation | Wind | etc. |
| Station A | Distance, etc. | 8:02am | 54F | 0.0in | 6.2mph | … |
| Station B | Distance, etc. | 8:04am | 53F | 0.0in | 6.9mph | … |
| Station A | Distance, etc. | 8:07am | 54F | 0.0in | 6.5mph | … |
| Station C | Distance, etc. | 8:09am | 56F | 0.1in | 7.9mph | … |
| Station A | Distance, etc. | 8:12am | 55F | 0.0in | 7.2mph | … |
| Station B | Distance, etc. | 8:14am | 54F | 0.1 in | 7.8mph | … |
While a detailed explanation of how to process these records is outside the scope of this article, there are some high-level considerations that merit mentioning. First, the result records represent exact weather observations at the specific time and place given in the report. However, that time and place is unlikely to be exactly the one requested in the query. Instead, the location details will be those of an actual station near your requested location. The same applies to the time. The reporting times will be within the time range that was requested, but they will not be any specific pattern or time increment.
Also the result is likely to contain reports from several nearby the stations. The requested number of stations and physical distance is configurable in the query. In fact, the same stations may report several times within the requested period. So your application must be prepared to consume these individual records and interpret them as needed for your use case.
As Spider Man famously says, “With great power comes great responsibility.” This is certainly true when it comes to raw weather data. If this is what you need, Visual Crossing Weather can provide exactly what you seek. If not, however, you likely want to let the Visual Crossing Weather engine handle the interpretation for you as discussed above in the Simple Query section.
Current Conditions
An valuable topic that is related to sub-hourly data is the feature known as current conditions. This feature uses the sub-hourly weather feed to provide, as closely as possible, the current weather conditions in a given location. It does so by returning the most recently reported conditions based on nearby stations. In the US and much of Europe our sub-hourly network typically reports data less than 15 minutes old and in some cases less than 5 minutes. So current conditions provides an ideal way to get the latest conditions for any point. You should be aware, however, that some international areas report data much less frequently. In the more extreme cases, weather reports can be delayed by a day or longer. Each query result is timestamped so that you can easily tell how recently the conditions were observed.
Summary
Visual Crossing Weather’s sub-hourly or minute-by-minute weather data functionality brings an entirely new level of weather analysis and opens up a new world of weather use-cases. You can now build tools that will tell you the weather conditions at the exact time of an accident based on the police report. You can design a tool that will help you understand the precipitation levels in a specific area when a storm drain overflowed.
You can even use these detailed historical weather conditions to determine the likely time period for other real-world events. For example, if a tree falling on the road was the catalyst for an accident, knowing the time that the storm came through the location and the period of peak winds would greatly help you narrow down the accident time window. This can be a huge value in forensic analysis, legal cases, insurance, and more.
Visual Crossing Weather is the only weather data provider that has made minute-by-minute data so easily accessible and affordable. This data was once only available to customers paying tens of thousands of dollars and willing to spend days or weeks processing it. Visual Crossing Weather has now made it available for everyone who can process a simple web query.
For more information and to get your own sub-hourly weather data, please sign up for your Visual Crossing Weather account!