ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market

Félix Barzin; Gonzague Yernaux; Wim Vanhoof

doi:10.1109/wi-iat59888.2023.00054

ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market

Félix Barzin, Gonzague Yernaux, Wim Vanhoof

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

109 Downloads (Pure)

Abstract

Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges.The discussion revolves around a concrete application, namely the automatic extraction of data concerning the Belgian real estate market. We introduce a real-time Web scraper called \scrimmo~and tailored to collect data from websites containing real estate classified ads. The tool is developed in a continuous iterative process and based on an innovative cloud architecture. The paper also briefly addresses the ethical aspects of Web scraping. By integrating insights from previous research and ethical guidelines, this study provides researchers with a comprehensive understanding of Web scraping and its potential benefits, while promoting responsible and ethical practices in data collection and analysis.

Original language	English
Title of host publication	Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023
Pages	335-338
Number of pages	4
ISBN (Electronic)	9798350309188
DOIs	https://doi.org/10.1109/wi-iat59888.2023.00054
Publication status	Published - Oct 2023
Event	The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology - Venise, Italy Duration: 26 Oct 2023 → 29 Oct 2023 https://www.wi-iat.com/wi-iat2023/index.html

Publication series

Name	Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023

Conference

Conference	The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology
Abbreviated title	WI-IAT 2023
Country/Territory	Italy
City	Venise
Period	26/10/23 → 29/10/23
Internet address	https://www.wi-iat.com/wi-iat2023/index.html

Keywords

Data analysis
Data extraction
Data gathering
Web crawling
Web scraping

Access to Document

10.1109/wi-iat59888.2023.00054

ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate MarketAccepted author manuscript, 261 KB
2023ScrImmoFinal published version, 333 KBLicence: Unspecified

Cite this

Barzin, F., Yernaux, G., & Vanhoof, W. (2023). ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market. In Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 (pp. 335-338). (Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023). https://doi.org/10.1109/wi-iat59888.2023.00054

Barzin, Félix ; Yernaux, Gonzague ; Vanhoof, Wim. / ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market. Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023. 2023. pp. 335-338 (Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023).

@inproceedings{18367766da4449e98f1cbee722b65a77,

title = "ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market",

abstract = " Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges.The discussion revolves around a concrete application, namely the automatic extraction of data concerning the Belgian real estate market. We introduce a real-time Web scraper called \scrimmo~and tailored to collect data from websites containing real estate classified ads. The tool is developed in a continuous iterative process and based on an innovative cloud architecture. The paper also briefly addresses the ethical aspects of Web scraping. By integrating insights from previous research and ethical guidelines, this study provides researchers with a comprehensive understanding of Web scraping and its potential benefits, while promoting responsible and ethical practices in data collection and analysis.",

keywords = "Data analysis, Data extraction, Data gathering, Web crawling, Web scraping",

author = "F{\'e}lix Barzin and Gonzague Yernaux and Wim Vanhoof",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023 ; Conference date: 26-10-2023 Through 29-10-2023",

year = "2023",

month = oct,

doi = "10.1109/wi-iat59888.2023.00054",

language = "English",

series = "Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023",

pages = "335--338",

booktitle = "Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023",

url = "https://www.wi-iat.com/wi-iat2023/index.html",

}

Barzin, F, Yernaux, G & Vanhoof, W 2023, ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market. in Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023. Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023, pp. 335-338, The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, Venise, Italy, 26/10/23. https://doi.org/10.1109/wi-iat59888.2023.00054

ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market. / Barzin, Félix; Yernaux, Gonzague ; Vanhoof, Wim.
Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023. 2023. p. 335-338 (Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023).

Research output: Contribution in Book/Catalog/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market

AU - Barzin, Félix

AU - Yernaux, Gonzague

AU - Vanhoof, Wim

PY - 2023/10

Y1 - 2023/10

N2 - Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges.The discussion revolves around a concrete application, namely the automatic extraction of data concerning the Belgian real estate market. We introduce a real-time Web scraper called \scrimmo~and tailored to collect data from websites containing real estate classified ads. The tool is developed in a continuous iterative process and based on an innovative cloud architecture. The paper also briefly addresses the ethical aspects of Web scraping. By integrating insights from previous research and ethical guidelines, this study provides researchers with a comprehensive understanding of Web scraping and its potential benefits, while promoting responsible and ethical practices in data collection and analysis.

AB - Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges.The discussion revolves around a concrete application, namely the automatic extraction of data concerning the Belgian real estate market. We introduce a real-time Web scraper called \scrimmo~and tailored to collect data from websites containing real estate classified ads. The tool is developed in a continuous iterative process and based on an innovative cloud architecture. The paper also briefly addresses the ethical aspects of Web scraping. By integrating insights from previous research and ethical guidelines, this study provides researchers with a comprehensive understanding of Web scraping and its potential benefits, while promoting responsible and ethical practices in data collection and analysis.

KW - Data analysis

KW - Data extraction

KW - Data gathering

KW - Web crawling

KW - Web scraping

UR - http://www.scopus.com/inward/record.url?scp=85182526364&partnerID=8YFLogxK

U2 - 10.1109/wi-iat59888.2023.00054

DO - 10.1109/wi-iat59888.2023.00054

M3 - Conference contribution

T3 - Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023

SP - 335

EP - 338

BT - Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023

T2 - The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology

Y2 - 26 October 2023 through 29 October 2023

ER -

Barzin F, Yernaux G , Vanhoof W. ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market. In Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023. 2023. p. 335-338. (Proceedings - 2023 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2023). doi: 10.1109/wi-iat59888.2023.00054

ScrImmo: A Real-time Web Scraper Monitoring the Belgian Real Estate Market

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

The 22nd IEEE/WIC International Conference on Web Intelligence and Intelligent Agent Technology