Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Technological Surveillance in Big Data Environments by using a MapReduce-based Method

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

For many years, the organizations had monitored the technological environments to anticipate changes with potentially positive or negative impacts on their business using technological surveillance process. However, the new Big Data scenarios turned the traditional tools and methods no longer sufficient. This paper proposes an automated technological surveillance method by using a map-reduce model to deal with Big Data scenarios divided into five processes: planning, collection, organization, intelligence, and communication. We implemented a system prototype to validate the proposed approach. It was developed in Python and Javascript, using ontologies for knowledge modeling, NoSql database to store and parallel processing of the publications. The system collected 2,918 publications, identified the monitored technologies, extracted the metadata, analyzed them, and generated charts for the stakeholders. In conclusion, the method demonstrated be feasible to automate the technology watch process in Big Data scenarious and dramatically reduced the workload involved when implemented by a system, offering a solid approach to automatically identify a set of technologies with increasing popularization in Web portals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.formobile.com.br

  2. http://www.furniturenews.net

  3. http://www.megamoveleiros.com.br

  4. http://www.woodbusiness.ca

  5. http://www.woodworkingnetwork.com

References

  1. Abe H, Tsumoto S (2009) Detecting temporal trends of technical phrases by using importance indices and linear regression. In: International symposium on methodologies for intelligent systems. Springer, Berlin, Heidelberg, pp 251–260

  2. ABNT Brazilian Association of Technical Standards. Guidelines for research, development and innovation management systems (R&D&I) [In Portuguese: Associação Brasileira de Normas Técnicas. Diretrizes para sistemas de gestão da pesquisa, do desenvolvimento e da inovação (PDI).] https://www.abntcatalogo.com.br/norma.aspx?ID=088796. Accessed 28 Oct 2020. Accessed 28 Oct 2020"/>

  3. AENOR Spanish Association for Standardization and Certification. Spanish Experimental Standard Rule UNE 166006 R&D&I: Technological Surveillance System [In Spanish: Asociación Española de Normalización y Certificación. Norma Española Experimental UNE 166006 Gestión de la I +D +i: Sistema de Vigilancia Tecnológica]. https://www.aenor.com/normas-y-libros/buscador-de-normas/une?c=N0059973. Accessed 28 OCt 2020

  4. Salgado Batista D, Guzmán Sánchez MV, Carrillo Calvet H (2003) Establishment of a scientific-technological surveillance system. Technological Forecasting [Original title in Spanish: Establecimiento de un sistema de vigilancia científico-tecnológica]. ACIMED 11(6):0–0

    Google Scholar 

  5. Dean J, Ghemawat S (2008) MapReduce: Simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  6. Ena O, Mikova N, Saritas O, Sokolova A (2016) A methodology for technology trend monitoring: the case of semantic technologies. Scientometrics 108(3):1013–1041

    Article  Google Scholar 

  7. Flick U (2009) Introduction to Qualitative Research [Original title in Portuguese: Introdução à Pesquisa Qualitativa]. Artmed, Porto Alegre

  8. Geum Y, Jeon J, Seol H (2013) Identifying technological opportunities using the novelty detection technique: A case of laser technology in semiconductor manufacturing. Technol Anal Strateg Manag 25(1):1–22

    Article  Google Scholar 

  9. Gil AC (2010) How to design research projects [Original title in Portuguese: Como elaborar projetos de pesquisa]. Atlas, Barueri

  10. Hakim AR, Djatna T (2015) Extraction of multi-dimensional research knowledge model from scientific articles for technology monitoring. In: 3rd international conference on adaptive and intelligent agroindustry (ICAIA). IEEE, pp 300–305

  11. León AM, Castellanos OF, Vargas FA (2006) Assessment, selection and relevance of software tools used in technological surveillance [Original title in Spanish: Valoración, selección y pertinencia de herramientas de software utilizadas en vigilancia tecnológica]. Ingeniería e investigación 26(1):92–102

    Google Scholar 

  12. Momeni A, Rost K (2016) Identification and monitoring of possible disruptive technologies by patent-development paths and topic modeling. Technol Forecast Social Chang 104:16–29

    Article  Google Scholar 

  13. Nam S, Kim K (2017) Monitoring newly adopted technologies using keyword based analysis of cited patents. IEEE Access 5:23086–23091

    Article  Google Scholar 

  14. Palop F, Vicente JM (1999) Technological Surveillance and Competitive Intelligence. Its potential for Spanish companies [Original title in Spanish: Vigilancia tecnológica e inteligencia competitiva: su potencial para la empresa española]. Cotec, Madrid

  15. Sánchez Torres JM, Palop Marro F (2002) Software tools for the practice in the company of Technological Surveillance and Intelligence [Original title in Spanish: ”Herramientas de software para la práctica en la empresa de la Vigilancia Tecnológica e Inteligencia Competitiva”], Evaluacioń Comparativa, 1© Edición, TRIZ. España

  16. Park H, Kim E, Bae KJ, Hahn H, Sung TE, Kwon HC (2011) Detection and analysis of trend topics for global scientific literature using feature selection based on gini-index. In: IEEE 23rd international conference on tools with artificial intelligence. IEEE, pp 965–969

  17. Pascal-Filho DSM, Macedo DDJ, Dutra ML (2021) A MapReduce-based Method for Achieving Active Technological Surveillance in Big Data Environments. 2nd EAI International Conference on Data and Information in Online Environments. Florianopolis, Brazil, DIONE

  18. PDIC2022 Industry Development Program of the Brazilian State of Santa Catarina for 2022: competitiveness with sustainability [In Portuguese: Programa de Desenvolvimento da Indu’́stria Catarinense 2022: competitividade com sustentabilidade]. http://www4.fiescnet.com.br/homepdic. Accessed 28 Oct 2020

  19. Sharma S, Gupta P (2015) The anatomy of web crawlers. In: International conference on computing, communication automation. IEEE, Greater Noida, India, pp 849–853

  20. Shiryaev AP, Dorofeev AV, Fedorov AR, Gagarina LG, Zaycev VV (2017) LDA models for finding trends in technical knowledge domain. In: IEEE conference of russian young researchers in electrical and electronic engineering (EIConRus). IEEE, pp 551–554

  21. Tomaél MI (2008) Information sources on the internet [Original title in Portuguese: Fontes de informação na internet]. EDUEL, Londrina

  22. Wei YM, Kang JN, Yu BY, Liao H, Du YF (2017) A dynamic forward-citation full path model for technology monitoring: An empirical study from shale gas industry. Appl Energy 205:769–780

    Article  Google Scholar 

  23. White T (2015) Hadoop: The definitive guide. O’Reilly, Beijing

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PGCIN, Federal University of Santa Catarina, Florianópolis, Brazil

    Daniel San Martin Pascal Filho

  2. Departament of Information Science, Federal University of Santa Catarina, Florianópolis, Brazil

    Douglas Dyllon Jeronimo de Macedo & Moisés Lima Dutra

Authors
  1. Daniel San Martin Pascal Filho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas Dyllon Jeronimo de Macedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moisés Lima Dutra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel San Martin Pascal Filho.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Filho, D.S.M.P., de Macedo, D. & Dutra, M.L. Technological Surveillance in Big Data Environments by using a MapReduce-based Method. Mobile Netw Appl 27, 1931–1940 (2022). https://doi.org/10.1007/s11036-022-01962-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-022-01962-2

Keywords

Search

Navigation