International Journal of Computer and Information Technology

Open Access Open Access  Restricted Access Subscription Access

Comprehensive Design Flow for Effective Data Processing and Visualization


Affiliations
1 Department of Electronics and Telecommunication Engineering, R V College of Engineering, Bangalore, India
 

Data is pervasive in the modern digital world. Data is continually being created, gathered, and analysed, from social media interactions to financial transactions. Businesses face both opportunities and challenges due to the data boom. On one hand, data may be a useful resource for companies looking to differentiate themselves from the competition, spur innovation, and enhance decision-making. However, the sheer amount and complexity of data can be daunting, making it challenging to glean insightful information and take timely action on it. Organisations need a strong and well-designed data processing design flow that can allow effective data exploration and creative visualization in order to meet this challenge. This work focuses on the complete data processing lineage and integrating that with various tools such as Talend and Power BI for data analytics to deliver diverse visualizations and insights which can cater businesses to stay ahead of the curve, providing a foundation for success in the ever-evolving digital landscape.

Keywords

Talend, Power BI, Data Analytics, Exploration
User
Notifications
Font Size

  • H. Ashok, S. Ayyasamy, A. Ashok and V. Arunachalam, "E-business Analytics through ETL and Self-Service Business Intelligence Tool," 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2020, pp. 747-751, doi: 10.1109/ICIRCA48905.2020.9183336.

  • W. Jijun, C. Yongqiu and C. Li, "Research on Multi-layer Power Enterprise Data Management Architecture Based on Big Data," 2022 IEEE 8th International Conference on Computer and Communications (ICCC), Chengdu, China, 2022, pp. 1194-1198,

  • Y. Mao, S. Huang, S. Cui, H. Wang, J. Zhang and W. Ding, "Multidimensional data distribution monitoring based on OLAP," 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China, 2020, pp. 298-302, doi: 10.1109/ITCA52113.2020.00070.

  • S. Yi and H. Ye, "A Practical Workflow in Cleaning Master Data," 2020 IEEE International Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), Exeter, United Kingdom, 2020, pp. 1311-1314

  • A. Kaminskyi and M. Nehrey, "Information Technology Model for Customer Relationship Management of Nonbank Lenders: Coupling Profitability and Risk," 2021 11th International Conference on Advanced Computer Information Technologies (ACIT), Deggendorf, Germany, 2021,pp. 234-237

  • J. Sreemathy, S. Priyadharshini, K. Radha, K. Sangeerna and G. Nivetha, "Data Validation in ETL Using TALEND," 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), Coimbatore, India, 2019, pp. 1183-1186, doi: 10.1109/ICACCS.2019.8728420.

  • M. Riasetiawan, F. Anggara, A. Ashari, S. Winardi and B. N. Prastowo, "Data Model and Analysis for Big Data Mapping and Management in the Energy Data Platform," 2021 International Conference on Data Science, Artificial Intelligence, and Business Analytics (DATABIA), Medan, Indonesia, 2021, pp. 1-6,

  • J. Sreemathy, I. Joseph V., S. Nisha, C. Prabha I. and G. Priya R.M., "Data Integration in ETL Using TALEND," 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2020, pp. 1444-1448, doi: 10.1109/ICACCS48705.2020.9074186.

  • J. Sreemathy, R. Brindha, M. Selva Nagalakshmi, N. Suvekha, N. Karthick Ragul and M. Praveennandha, "Overview of ETL Tools and Talend-Data Integration," 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 2021, pp. 1650-1654, doi: 10.1109/ICACCS51430.2021.9441984.

  • P. S. Diouf, A. Boly and S. Ndiaye, "Variety of data in the ETL processes in the cloud migration and validation : State of the art," 2018 IEEE International Conference on Innovative Research and Development (ICIRD), Bangkok, 2018, pp. 1-5.

  • M. Balakrishnan, N. M, K. Ramya and S. K, "Cloud Computing based Data Validation and Migration in ETL using Talend," 2022 6th International Conference on Electronics, Communication and Aerospace Technology, Coimbatore, India, 2022, pp. 1349-1355, doi: 10.1109/ICECA55336.2022.10009262.

  • R. Akbar, Wahyudi, J. Rahmadoni and H. A. Lubis, "The Implementation of Business Intelligence on Visualisation of Transaction Data Analysis using Dashboard System Case Study: XYZ Convenience Store," 2022 International Symposium on Information Technology and Digital Innovation (ISITDI), Padang, Indonesia, 2022, pp. 42-49, doi: 10.1109/ISITDI55734.2022.9944494.

  • M. Zhang, J. Zhou, Q. Wang and X. Zhang, "Data Quality Management for Big Data: A Review," in IEEE Access, vol. 7, pp. 70102-70114, 2019


Abstract Views: 86

PDF Views: 35




  • Comprehensive Design Flow for Effective Data Processing and Visualization

Abstract Views: 86  |  PDF Views: 35

Authors

Samudyata A
Department of Electronics and Telecommunication Engineering, R V College of Engineering, Bangalore, India
P Nagaraju
Department of Electronics and Telecommunication Engineering, R V College of Engineering, Bangalore, India

Abstract


Data is pervasive in the modern digital world. Data is continually being created, gathered, and analysed, from social media interactions to financial transactions. Businesses face both opportunities and challenges due to the data boom. On one hand, data may be a useful resource for companies looking to differentiate themselves from the competition, spur innovation, and enhance decision-making. However, the sheer amount and complexity of data can be daunting, making it challenging to glean insightful information and take timely action on it. Organisations need a strong and well-designed data processing design flow that can allow effective data exploration and creative visualization in order to meet this challenge. This work focuses on the complete data processing lineage and integrating that with various tools such as Talend and Power BI for data analytics to deliver diverse visualizations and insights which can cater businesses to stay ahead of the curve, providing a foundation for success in the ever-evolving digital landscape.

Keywords


Talend, Power BI, Data Analytics, Exploration

References