Using web scraping for real estate price analysis

Aleksandar Tsonev, Valentina Dyankova, Yusuf Yusufov

Abstract


In the present paper, a survey is being made of the areas, in which the use of Web scraping is necessary for adaptation to dynamic changes, demanding processing and generalizing of large volume of data from different web sources. An exemplary implementation for Web scraping is presented for price analysis of the real estate market. The technology that gives possibility of integration of the intended activities in development of the application has been reviewed. The overall procedure in the researching, development and realization of the proposed application can be used as a good practice in adaptation of the knowledge and skills of the graduating students for the requirements, dynamics and organization of the work in a software company as well as for the professional co-operation between the academic lecturers and representatives of the software business.

Keywords


web scraping; data extraction; databases; Selenium; Java; SpringBoot; Hibernate

Full Text:

PDF

References


Erdinç Uzun, A Novel Web Scraping Approach Using the Additional Information Obtained From Web Pages, IEEE Access, volume 8, 2020, ISSN: 2169-3536.

Jay M. Patel, Getting Structured Data from the Internet: Running Web Crawlers/Scrapers on a Big Data Production Scale, Apress, 2020, ISBN 978-1484265758.

Lusiana Citra Dewi, Meiliana, Alvin Chandra, Social Media Web Scraping using Social Media Developers API and Regex, Procedia Computer Science, Volume 157, 2019, Elsevier.

Junk, Yoonwon; Lee, Yoon Kyung; Hahn, Sowon, Web-scraping the Expression of Loneliness during COVID-19, Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 43, 2021, e-ISSN 1069-7977.

Saram Han, Christopher K. Anderson, Web Scraping for Hospitality Research: Overview, Opportunities, and Implications, Cornell Hospitality Quarterly, Volume 62, Issue 1, 2020.

Elena Hernández-Nieves, Álvaro Bartolomé del Canto, Pablo Chamoso-Santos, Fernando de la Prieta-Pindato, Juan M. Corchado-Rodríguez, A Machine Learning Platform for Stock Investment Recommendation Systems, Distributed Computing and Artificial Intelligence, 17th International Conference, Italy, 17th–19th June, 2020.

Moaiad Ahmad Khder, Web Scraping or Web Crawling: State of Art, Techniques, Approaches and Application, International Journal of Advances in Soft Computing and its Applications, Vol. 13, No. 3, November 2021.

Tomasz Jach, Web Scraping Methods Used in Predicting Real Estate Prices, Advances in Computational Collective Intelligence, 13th International Conference, ICCCI 2021, Kallithea, Rhodes, Greece, September 29 – October 1, 2021.

Pete Houston, Instant jsoup How-to, Packt Publishing, 2013, ISBN 978-1782167990.

Ryan Mitchell, Instant Web Scraping with Java, Packt Publishing, 2013, ISBN 978-1849696883.

Boni Garcia, Hands-On Selenium WebDriver with Java: A Deep Dive into the Development of End-to-End Tests, O'Reilly Media, 2022, ISBN 978-1098110000.

Pallavi Sharma, Selenium with Java – A Beginner’s Guide: Web Browser Automation for Testing using Selenium with Java, BPB Publications, 2022.

Kevin Sahin, Java Web Scraping Handbook, Leanpub, 2018-07-26.




DOI: http://dx.doi.org/10.5281/zenodo.7294898

Refbacks

  • There are currently no refbacks.
We use cookies.