Data Quality Management

 
Data Quality Management

Data Quality Management

EII Task force

Summary

As the scale and diversity of data grows in the digital arena, the complexities of data driven engineering grow multifold with it. The last several years have brought forth several new technologies to service this need such as grid systems and web services. However, a fundamental underpinning of the success of these technologies resides in the quality of data that they can provide. Often the failure of a technology is attributed to its functionality when the real problem lies in the quality of data it uses and subsequently produces. Estimates from industry (Gartner, 2007) indicated a financial loss due to poor data quality in access of $611 billion in US alone, whereas, the investment in data quality tools and products, remains under $600 million (Forrester Research, 2008), indicating a clear and evident need to create a better understanding of the issues, challenges and solutions for data quality management.

DURATION

July 2008 – July 2009

TASKFORCE COORDINATOR/S

Shazia Sadiq, Ke Deng

School of Information Technology and electrical Engineering, The University of Queensland

{shazia, dengke}@itee.uq.edu.au

SENIOR RESEARCH ADVISOR/MENTORS

Prof. Xiaofang Zhou

School of Information Technology and electrical Engineering, The University of Queensland

zxf@itee.uq.edu.au

Prof. Graeme Shanks

Department of Information Systems

University of Melbourne

gshanks@unimelb.edu.au

TASKFORCE TEAM MEMBERS

Prof. Xuemin Lin

School of Computer science and Engineering

The University of New South Wales

lxue@cse.unsw.edu.au

Dr. Marta Indulska

UQ Business School, The University of Queensland

m.indulska@business.uq.edu.au

Dr. Rosanne Price

Faculty of Information Technology

Monash University

rosanne.price@infotech.monash.edu.au

Dr. Gabriel Fung

School of Information Technology and electrical Engineering, The University of Queensland

pcfung@itee.uq.edu.au 2

OTHER TARGETED PARTICIPANTS

The task force will put a concerted effort towards outreach with industry (see [11] below), including both the user organizations as well as data quality vendors. The task force members have already approached and identified some participants mentioned below, but this list is expected to increase.

Vendors: Informatica, DatFlux, Trillium Software, Veda Advantage

Users: Shell Australia, MBF, Medicare, Telstra

Secondly, the taskforce will engage with international experts in research and academia to develop deep understanding of issues and challenges, as well as to validate key findings solicited from outreach activities. This engagement will take place through a comprehensive survey of the existing literature as well as a series of interviews on phone or face-to-face where possible. Experts identified so far include the following, but this list is also expected to increase.

Experts: Divesh Srivastava (AT&T Labs); Wenfei Fan (University of Edinburgh); Richard Wang (Information Quality Group, MIT); Thomas Redman (Navesink Consulting); Members of the International Association for Information and Data Quality (iaidq.org)

TASKFORCE SIGNIFICANCE

The area of data quality is being studied from three different perspectives: Organizational: Development of data quality objectives for the organization and strategies to establish the people, processes, policies, and standards required to manage and ensure the data quality objectives are met; Architectural: The technology landscape required to deploy developed data quality management processes, standards and policies; and Computational: Effective and efficient tools & techniques required to meet data quality objectives

However, we believe that a holistic data quality management solution must bring together the three perspectives in a complementary rather than competitive manner. The EII community is characterized by diverse strengths and interests. Through the efforts of this task force we hope to bring together the three perspectives, and thereby will create synergies between the different communities, as well as create a better understanding and appreciation of holistic approaches.

Further, bringing together diverse teams to work towards a common complex problem will set an example beyond the scope and lifetime of this task force and demonstrate leadership of the network in this regard.

TASKFORCE SCOPE

The objectives of the task force are summarized as below:

  • Conduct a thorough review of data quality management research literature

  • Engage with industry and create an understanding of real life data quality problems and approaches to data quality management through a compilation of case studies

  • Survey the data quality product market, and where possible contact vendors to gain first hand knowledge of product portfolios and capabilities

  • Prepare a comprehensive report on the topic based on the above findings

  • Develop teaching materials from the report

  • Organize a workshop in conjunction with DASFAA (April, 2009) to attract further interest in the field and provide visibility to the topic as well as the work accomplished above

SPECIFIC MILESTONES, EVENTS or ACTIVITIES

Survey of research literature, development of case studies and vendor analysis Jul 2008 – Dec 2008
Meeting of Minds Workshop Aug 2008
Compilation of data and preparation of report Jan 2009 – Mar 2009
International Workshop on Data Quality in conjunction with DASFAA 2009 Apr 2009
Development of teaching material Apr 2009 – Jul 2009
  
NICTA is Australia's new Information Communication Technology (ICT) centre of excellence Logo: Griffith University Logo: The University of Queensland Logo: CSIRO Logo: Victoria University Logo: University of New South Wales Logo: Maquarie University Logo: University of Melbourne Logo: University of Swinburne Logo: University of Newcastle Logo: University of Sydney Logo: University of South Australia
Logo: Queensland University of Technology Logo: University of Wooloongong Logo: The University of Technology, Sydney Logo: Monash University Logo: Edith Cowan University Logo: Australian National University Logo: SAP Logo: Microsoft