Fundació La Caixa: data collection via open data and scraping


To know how the weather and the exhibitions and activities of the main competing museums impact on the number of visitors to the museum.


Automate the process with scraping techniques from the main websites of museums in the country and obtaining meteorological data from AEMET.


To have data to make the analysis and to be able to improve the proposals.


In any sector, knowing why there is more or less public traffic to a point of sale is not an easy task. Normally, we think that it depends on the products offered, the campaigns carried out, the price, the promotion of the day… but there are also other variables that are not in our hands, such as the calendar, seasonality, the weather or the action of the competition. Fundació La Caixa set out to incorporate into its analytical model the data on exhibitions and activities offered by competing museums and the open weather data offered by the AEMET.


AEMET OpenData is a REST API developed by AEMET that allows the dissemination and reuse of the Agency’s meteorological and climatological information.


Web scraping refers to the process of extracting content and data from websites using software. We were able to identify a set of data common to all museum websites that were relevant to the analytical model: Title and Subtitle of the Exhibition or Activity, Start and end dates, web url of the activity/exhibition.

On a monthly basis, the process is automatically run and made available to the analysis team.