Accepted answer

I noticed one thing.

It seems true that:

"when I use custom search option, csv file that automatically generated by the website is not updated".

But actually it is not true. It is updated, but the maximum "custom data range" seems to be 1 year.

For example, if you set from 2013-09-23 to 2017-10-23 the .csv(.json) generated has max the data of 1 year (in this example from 22/10/2016 to 21/10/2017).

You can better notice this if you play with the "extremes".

For example with:
  • first element: {"date": "2014-10-23", "count": 212730, "end": "2014-10-23"}
  • last element: {"date": "2013-10-24", "count": 163094, "end": "2013-10-24"}

if you change with:
  • first element: {"date": "2014-10-24", "count": 215105, "end": "2014-10-24"}
  • last element: {"date": "2013-10-25", "count": 168018, "end": "2013-10-25"}

Or with:

will be again :

  • first element: {"date": "2014-10-23", "count": 212730, "end": "2014-10-23"}
  • last element: {"date": "2013-10-24", "count": 163094, "end": "2013-10-24"}

So, in order to get the data of the last 5 years you could do:

import subprocess
tokenDataToday= today.split("-")
url= ""

while year <= interestedYears:
     yearStart= str(int(float(tokenDataToday[0]))-year)
     #print("dateStart: " + dateStart)
     #print("dateEnd: " + dateEnd)
     cmd = 'curl -O ' + tmpUrl
     args = cmd.split()
     process = subprocess.Popen(args, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     stdout, stderr = process.communicate()
     year = year+1


Mozilla provides a simple REST API to get the stats, so you don't need to use Selenium.

With the requests module:

url = ""
data = requests.get(url).json()

To select the range, simply update the dates in the URL.

But if you are still willing to scrap the chart with selenium:

dates = driver.execute_script("return Highcharts.charts[0].series[0].xData");
users = driver.execute_script("return Highcharts.charts[0].series[0].yData");
downloads = driver.execute_script("return Highcharts.charts[0].series[1].yData");

Related Query

More Query from same tag