Visual Crossing’s weather Query Builder makes it easy to build weather API queries and download result data for any date or time period. However, in some use cases you need to move beyond a web-based interface and automate your weather data retrieval. While this is easy to do if you are a coder or scripter, doing so can be a challenge for those who are less code savvy. Luckily, the standalone tool cURL can give many of the same benefit for those who are not knowledgeable in computer languages such as Java or Python or even for those who just want to find the simplest way to do the simple task of retrieving weather data. For those who do not want to code and just need some basic automation, cURL can often provide exactly what you need without requiring you to break out a code development environment.
In this article will show you how to automate weather data retrieval using the common command-line tool cURL.
What is cURL?
If you are using Unix or Linux, you may already be familiar with cURL. cURL is a very useful and powerful command-line tool that wraps the programming library libcurl. Short for “Client URL,” cURL is an easy-to-use tool that allows anyone to automate web queries and fetch result data. Most Unix and Linux systems as well as all modern Macs already have cURL installed and ready for you to use. If you type “curl” at a command prompt and find it unavailable and if you have admin privileges, you can install it from any standard repository. If you are not an administrator, your local system administrator will be able to configure cURL for you in a matter of minutes.
Any recent version of Microsoft Windows (Windows 10 build 1803 and later, for example), including Windows 11, come with cURL already installed. To ensure you have it already, simply open a command prompt (or PowerShell if you are more comfortable in that environment), type “curl,” and hit enter. If you get a help message specifically for the cURL command, you are all set. If instead you get and error about your command not being recognized or if you are using some alternate OS platform, it is easy to obtain cURL for just about any system. There are many web-based builds and installations available, and one of the most common sources is here: https://curl.haxx.se/download.html. Simply select your OS and follow the appropriate download link. (Note that this link is to a 3rd-party site over which Visual Crossing has no control. Although we use this site ourselves, as with any internet download, please scan all executables for viruses and make sure to resolve any other concerns before installing.)
How can I create a query URL to use in my cURL command?
Before we can use cURL, we need to build our weather query URL. This URL fully describes our weather query and can be passed directly to the Visual Crossing API servers. Luckily, there are two easy ways to do this, and one is entirely done within a web user interface. We can write our query either entirely in Visual Crossing Weather’s web-based Query Builder interactive interface or by constructing it manually as a string. The easiest option is to use the Query Builder. To get started, log into Visual Crossing’s Query Builder, configure your locations and date range all within the web UI. If you have not used Query Builder, you can read our guide that will walk you through how to use it step by step.
After we enter our location, Query Builder’s initial view will be a table of data that is the result of our weather query. If we just wanted this data for analysis in an external tool, we could stop here and download the data as a CSV or Excel file. (Simply engage the download wizard by clicking on the Download button above and to the right of the grid.) However, in this exercise, we are simply using Query Builder to generate a query URL for us to run in cURL. So, we now switch to the API view to see the exact query URL that Query Builder used to fetch the weather result data. We switch this view by clicking on the API button in the toolbar near the top of the window. We can copy this query URL by clicking on the Copy button on the right. Once we have this query URL, we can run it in any web browser to test it. (Note that this is a good idea and great debugging technique when creating and modifying query URLs.) We can also use this URL in a query tool such as cURL.
In order to make the output of our query URL more readable, we will change the “Output content type” dropdown on the left to be “CSV” instead of “JSON”. While JSON output is commonly used when writing code and in some advanced analysis tools, CSV data can be used in almost any environment and has the added benefit of being easily readable by a human.
The more astute observer will note that there is a drop-down menu on the left that allows Query Builder to customize the URL for specific use cases such as in Java or Python code. One of those options is cURL. You could use that generated cURL command directly in a script and skip the remainder of this article. However, since the purpose of this article is to learn more about using cURL manually, keep reading if you want to understand the details.
It is useful to note that the parameters in this generated URL can be easily edited either by changing the query settings in Query Builder or by editing this URL string. For example, you can easily change the date range, weather units, specific weather elements returned, and much more. For detailed information on how to edit these parameters directly in the query URL, please see our query API documentation.
One final key thing to note about the query URL is that it contains your Visual Crossing Weather API Key. For authentication purposes, your API Key must appear in every query URL. If you accidently omit it or make an error copy and pasting it, you will get an error response from our servers. So, please make sure to always include it, and if you are having problems using a given query URL, make sure to check for an error message result by pasting the URL into a web browser as suggested above. In nearly all cases, the returned error message will tell you exactly the cause of the error.
How do I fetch the weather data using cURL?
Now that we have our weather query URL, it is very easy to run it in cURL using the command line. While cURL command lines can become somewhat complex if you need to make specialized queries, for our first execution, we will use the easist possible command line. We will simply put the URL after the cURL command. The only adjustment required is to put the URL in quotes since some URLs will include special characters that can confuse command prompt environments. (Note that if you are on a platform other other than a standard Linux/Unix shell or a Windows command prompt, you may need to adjust the quoting mechanism to match your environment. However, if you are working in a custom environment, you almost certainly know more about using it than this article.)
So, here is the most basic example of using cURL to query forecast data for Washington, DC. Pasting it into your command prompt should give you a dump of CSV data back in response. (Note that if you want use the example below directly, you will to change YOUR_API_KEY to be your actual Visual Crossing Weather API Key. You can find it by logging into your Visual Crossing Weather account and clicking on the Account button in the upper right.)
curl "https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/Washington%2C%20DC?unitGroup=metric&include=days&key=YOUR_API_KEY&contentType=csv"
While you could manually copy this result data from the command prompt window into a text file, that would be tedious and error prone. You have much better uses for your time, and that is what automation tools such as cURL are all about. You can get the query results directly output into a file using cURL’s “-o” parameter flag (which is a shortened form and synonym of “–output”). This parameter tells cURL to output the query results to a file with a specific name instead of dumping it to the console window. This is very handy to use when using cURL to make a one-off query, but it is essential when running cURL from a script since it is impossible to copy-and-paste results as they rapidly scroll by.
So, our final cURL command-line has the form:
curl -o <OUT_FILE> "<VISUAL_CROSSING_WEATHER_QUERY_URL>"
Simply replace <OUT_FILE> with the name of the results file that you wish to created and <VISUAL_CROSSING_WEATHER_QUERY_URL> with the URL for your query that we created above.
In practice, based on the query example above, we would use a final command like this:
curl -o WeatherOutputFile.csv "https://weather.visualcrossing.com/VisualCrossingWebServices/rest/services/timeline/Washington%2C%20DC?unitGroup=metric&include=days&key=YOUR_API_KEY&contentType=csv"
Now that I can run query in cURL, what’s next?
Now that you can run a single weather query URL using cURL, you have the key building block to build weather query scripts of all types. For example, if you want to load a database with weather data, you can easily to so by creating a script containing multiple cURL commands each running a query URL, perhaps one query for each location or date range. You can then use ETL scripting for your database to import the data.
Another use would be if you want to periodically refresh a dataset automatically as part of an analysis process. You can use Visual Crossing Weather’s dynamic date feature to create a query URL that always returns data for “yesterday” or “nextweekend” or even the “lastNdays” where N is any value useful for your analysis. Powerful queries like these can be easily scripted using cURL. The best part is that they return updated data every time you run them.
In short, once you understand the basic format of a weather query URL and know how to use a tool such as cURL to script making that query, you have the power to automate hundreds of weather data tasks. You are limited only by your use case needs and your imagination.
Questions or need help?
If you have a question or need help, please post on our actively monitored forum for the fastest replies. You can also contact us via our support site or drop us an email at support@visualcrossing.com.