Topic

In Vitro & In Vivo: Techniques for the Evaluation Phase in Information Technology Research

This workshop is intended for a wide range of researchers in disciplines related to Information Technology and Computing, including Computer Science, Information Systems and Software Engineering.  In all these disciplines it is common for some artifact to be developed and used, with a range of artifacts including algorithms, software programs, software systems, methodologies, management interventions and organizational information systems. The emphasis placed on the evaluation phase can vary with the type of work done, with perhaps less emphasis on evaluation techniques in Computer Science and more on building the artifact. Conversely, in Information Systems, there is likely to be less emphasis on building the artifact, which may in fact have been developed by someone else, and more on evaluating the system in use  with human users. However, evaluation in some form is still important in nearly all our work. Further, experience from reading theses, journal articles and applications for research funding suggests that in many cases the evaluation phase is done rather poorly. In this workshop the two presenters will first give an overview of the research and evaluation lifecycle and the many types of evaluation techniques available, including simulation, laboratory experiments, fields studies, usability studies and case studies. Then Professor Gregor will focus more on “In Vitro” or controlled techniques, including experiments, and Professor Shanks on “In Vivo” or real-life studies, including case studies. Group exercises may be included. The expected outcome is more informed researchers who can better design and carry out full programs of investigation involving Information and Communication Technologies.

About Shirley Gregor

ANU Endowed Chair in Information Systems, Australian National University, Canberra, Australia

Email: shirley.gregor@anu.edu.au  

Web: http://ecocomm.anu.edu.au/people/info.asp?Surname=Gregor&Firstname=Shirley  

Shirley Gregor is the ANU Endowed Chair in Information Systems at the Australian National University, Canberra, where she heads the National Centre for Information Systems Research and is Head of the School of Accounting and Business Information Systems. Professor Gregor’s current research interests include the adoption and strategic use of information and communications technologies, intelligent systems and human-computer interface issues, and the theoretical foundations of information systems. Dr Gregor has led several large projects in the e-commerce area funded by the Meat Research Corporation, the Department of Communications, Information Technology and the Arts, and the Australian Research Council.  Professor Gregor spent a number of years in the computing industry in Australia and the United Kingdom before beginning an academic career. She obtained her Ph.D. in Information Systems from the University of Queensland. Dr Gregor’s publications include 4 edited books, 15 book chapters and over 80 papers in conferences and journals such as Management Information Systems Quarterly, International Journal of Electronic Commerce, International Journal of Human Computer Studies, International Journal of Electronic Commerce, European Journal of Information Systems and Information Technology & People. Professor Gregor was inaugural President of the Australasian Association of Information Systems 2002-2003 and is currently Vice-President of the Australian Council of Professors and Heads of Information Systems.  Professor Gregor was made an Officer of the Order of Australia in the Queen’s Birthday Honour’s list in June 2005 for services as an educator and researcher in the field of information systems and in the development of applications for electronic commerce in the agribusiness sector. In 2005 she was also elected as Fellow of the Australian Computer Society.

About Graeme Shanks

Professorial Fellow, Information Systems, The University of Melbourne, Australia

Email: gshanks@unimelb.edu.au   

Web: http://www.dis.unimelb.edu.au/people/academic/graeme_shanks.html   

Graeme Shanks is a Professorial Fellow in the Department of Information Systems at the University of Melbourne. He has also held academic positions at Monash University and Chisholm Institute of Technology, Australia. Professor Shanks's current research interests include data quality, decision support systems, identity management, conceptual modelling, and the implementation and impact of enterprise and inter-organisational systems. Dr. Shanks has led a number of research projects funded by the Australian Research Council, SAP, Novell and Telstra. Prior to becoming an academic, Professor Shanks worked for a number of years as programmer, programmer-analyst and project leader in several large organizations. He obtained his PhD in Information Systems from Monash University. Dr. Shanks has published over 100 papers in journals and conferences including Journal of Information Technology, Information Systems, Information Systems Journal, Communications of the ACM, Journal of Strategic Information Systems, Information and Management and Requirements Engineering Journal. Professor Shanks was a member of the Australian Research Council College of Experts in 2004 and 2005. He was elected as a Fellow of the Australian Computer society in 2004.