My Research Agenda
Over the years, I have developed, refined, and evolved my research agenda to address the ongoing issues and challenges (e.g., digital equity in education) in the field of educational technology and the immediate needs of my graduate students and academic program area I serve. While some of my research areas have remained constant over the years, others have progressed into different avenues based on new research methods I have learned over the years (e.g., meta-analysis) and others have emerged as a necessity to effectively teach and mentor in the constantly changing educational technology program (e.g., professional competencies). My research agenda has largely been shaped by my professional goal: the meaningful integration of information and communication technology (ICT) for the improvement of educational outcomes. This broad goal manifests itself into a few separate, but interrelated research themes that include: 1) effective design, development, utilization, and evaluation of theory-inspired, technology-enhanced learning environments; 2) appropriate teaching practices and instructional strategies for computer and information science education; 3) operationalization and measurement of technology integration in education, particularly focusing on the factors that facilitate and hinder technology integration in formal educational settings; 4) the study of the competencies of professionals in the broad field of educational technology, and 5) in more recent history, the practices and strategies to support effective online teaching and learning situations. My focus is on answering the following five broad research questions:
1. How do we design, develop, implement and evaluate theory-inspired, technology-enhanced learning environments for diverse learners and settings?
2. What are the effective and ineffective teaching practices and instructional strategies used in computer and information science education?
3. How do we operationalize and measure technology integration in formal educational settings, and which factors facilitate or hinder this integration?
4. What are the core competencies of professionals working across contexts in the broad field of educational technology, and how should we develop and validate these competencies?
5. What are the effective practices and strategies to support diverse learners in online teaching and learning environments, and how can we use this knowledge to assist aspiring online educators?
During the past ten years, my body of scholarship has largely focused on these five questions while balancing the interests of my doctoral students. At heart, I identify myself as a “designer” of research methods, instruments, instructional solutions and materials, and technology-enhanced learning environments. I use a wide variety of research methods to answer my research questions. I employ traditional experimental design research methods for testing many of my instructional designs and innovations in technology-enhanced learning environments. I use classical and modern test theory to establish measurement systems to inform my research and the research of others, using such procedures as exploratory factor analysis, confirmatory factor analysis, and more. I have employed literature synthesis and meta-analysis procedures to synthesize across primary studies on complex research questions. I have also used some more sophisticated data analysis procedures for analyzing larger data sets, including multi-level modeling and structural equation modeling techniques. I have also used qualitative techniques such as the constant comparative method, case study method, and phenomenology to answer questions related to deeply understanding situations and experiences.
Early in my academic career, I spent a great deal of time and effort to design, develop, utilize, and evaluate innovative information and communication technologies for educational purposes. This path led me to create and study a wide array of technological innovations, including games and simulations, multimedia learning environments, time-compression algorithms, ICT literacy tools, and even the application of ePortfolios in teacher education programs. In fact, my dissertation work, which has been cited by the well-known Wired magazine, was an experimental study systematically evaluating the use of time-compression algorithms in multimedia learning environments framed by the Cognitive Theory of Multimedia Learning (CTML). Much of this body of work used experimental and quasi-experimental research methods to examine aspects of these technologies using a cognitive theoretical perspective with students in both K-12 and postsecondary environments. I used a combination of between-subject and within-subject designs to address my research questions.
I also worked and continue to work on developing a wide array of measurement systems to serve as the dependent measures in my studies using classical and modern test theory. Some of these instruments have been translated into different languages for research and practice purposes. As the field of educational technology continued to evolve, so too did my approach and research methods to answer my questions. Although I was trained in primarily quantitative research methods, I have also adopted mixed-methods and qualitative research methods labeled as design and development cases to extend my expertise and ability to document the efficacy of technology-enhanced learning environments.
In more recent history, I have learned the fundamentals of meta-analysis, which I have used to answer burning questions that I previously was unable to address like identifying the overall effects of the flipped classroom instructional strategy on student learning outcomes. This strand of research is closely aligned with my teaching interests since I teach courses in introductory and advanced instructional design, educational software development, statistical research methods, and multimedia resource development. Because of this focus, I am able to use these experiences and knowledge in my classroom and when mentoring students. For instance, I recently graduated my first doctoral student who used meta-analysis for his dissertation which resulted in a best paper award, and have mentored another doctoral student using the design and development case study research approach. Also, my near completed federal grant from the National Institutes of Health (NIH) was based on my knowledge of design, development, and evaluation of technology-enhanced learning environments to develop a Massive Open Online Course (MOOC) for teaching diverse students about power and sample size analysis for multilevel and longitudinal research design in a fully online environment. These experiences have shaped my approach to scholarship and my perspectives on how to mentor doctoral students.
Dating back to my undergraduate honors thesis and my bachelor’s degree in computer and information sciences, I have had a long-standing interest in the quality teaching practices and instructional strategies used in computer and information science education in both K-12 and postsecondary environments. Prior to arriving at UF, I served as an instructor of computer science both at a community college and a university in Florida, teaching mostly introductory and intermediate computer programming and software development courses to undergraduate students majoring in computer science or information systems. During this period, I even hosted high school computer programming competitions and camps for students interested in learning the art and science of computer programming. Unfortunately, the evidence we have from empirical research shows that students at all ages struggle with learning computer programming concepts, particularly novice students taking their first computer programming course in a post-secondary institution.
Since introductory computer programming courses are gateway courses to the field of computer science, this area of research is a major concern both nationally and internationally. Thus, much of my research in the past decade has surrounded the effective teaching practices and instructional strategies used in computer programming coursework. In fact, my first scholarly publication was on teaching object-oriented computer programming using Visual Basic .NET. Since then, the integrated development environment technologies and programming languages available have continued to, and so too has my research agenda on these pressing issues. For instance, in more modern times, I have published two meta-analytic studies on pair-programming as an instructional strategy in computer programming courses, and block-based versus text-based computer programming environments used with novice programmers. The later publication earned a best paper award from AERA’s special interest group on Computer and Internet Application in Education at the 2019 conference in Toronto, Canada. Further, we are presently executing the later stages of a meta-analysis on student learning outcomes and retention in online versus face-to-face computer programming courses in both K-12 and higher education.
This body of research has been the stimulus for several of the National Science Foundation (NSF) grant programs that I have applied to over the years. While I am not always funded, I still continue to push the field and contribute to the discourse surrounding computer and information science education by my scholarly discourse, grant writing efforts, and serving as a reviewer for the NSF on computing education programs. One practical example of this work is that I am working with a team of researchers from UF to offer a new graduate certificate program in computer science education for practicing in-service teachers. Florida is in dire need of more qualified computer science teachers in K-12 schools. This academic program addresses a real need in our state while simultaneously using much of this research to inform the design and delivery of the program. This program will continue to lead to additional important work in computer science education, and hopefully, lead to securing national funding to scale and expand the coursework into formal degree programs. Our long-term goal is to establish both a master’s degree and doctoral degree concentration in computer science education to meet the needs of research and practice in this growing area of interest. Along these lines, I have also offered my time to the state of Florida to develop and validate the state assessment for certifying K-12 teachers to teach computer science. I have also provided professional development opportunities to dozens of teachers looking to add the K-12 computer science certification to their license. I am also proud to report that I have mentored several doctoral dissertations focusing on different aspects of computer and information science education, such as the newly established Advanced Placement (AP) Computer Science Principles course or the AP CSA course, which is equivalent a college-level computer programming course.
My research agenda surrounding the operationalization and measurement of technology integration in education dates back to my doctoral studies at the University of South Florida (USF), and my strong interest in ensuring the meaningful integration of ICT into formal educational settings. I started early in my doctoral studies on project involving technology integration particularly in public K-12 schools in Florida, and how to operationalize and measure this moving target as a construct of research and practice. This body of scholarship has also been perhaps my most influential work in terms of scholarly citations and impact on the field. My first project involved the issue of digital equity in Florida’s K-12 public schools, or what others and I have framed as the digital divide in schools. As a team, my colleagues and I have developed a conceptual framework to explain the nature of the digital divide in schools using large-scale datasets involving all public schools in the state of Florida, focusing on key outcome measures (e.g., technology access and usage) in relation to the socio-economic status (SES) of the schools.
This body of work has been influential in examining issues in Florida public schools related to digital equity such as access, use, and empowerment of teachers and students. Several researchers have adopted this conceptual framework used to explain the digital divide within formal educational environments. Using this conceptual framework, we used a combination of classical and modern measurement theory, and multilevel models to examine longitudinal data from Florida schools related to technology integration. Additionally, this research has resulted in media attention from both local and national (e.g., Education Week) news agencies, and an interview on National Public Radio (NPR) in 2013 about the digital divide and its evolving nature in formal K-12 school settings. I continue to work in this important area with recent articles, and handbook chapter about our conceptual framework to be published in the Handbook for Educational Communications and Technology, the authoritative handbook in the field of educational technology, sponsored by AECT.
As another example, I worked on the design, development, and validation of the Student Tool for Technology Literacy (ST2L), a performance-based assessment to measure middle school students’ ICT literacy in a simulated software environment. The ST2L was commissioned by the Florida Department of Education (FLDOE) to meet federal reporting requirements based on NCLB legislation. I am proud to say that the ST2L has been used by more than 180,000 students in the state of Florida since its release in 2008. The aggregate data have been used by policy-makers and administrators to make educational technology decisions in the state, teachers to make instructional decisions in the classroom, and to meet federal reporting requirements using robust, valid and reliable data. I continue to work in these areas employing advanced research methods including methods like path analysis, exploratory and confirmatory factor analysis, multilevel and longitudinal models, and multivariate statistical methods. I have also had the pleasure of mentoring several doctoral graduates at UF with deep interests in these areas, such as measuring information and communication technology literacy, or meeting the increasing technological demands placed on both K-12 students and teachers.
My most recent scholarly area has evolved from my teaching, mentoring, and advising interests and responsibilities in the broad field of educational technology and my responsibilities to my academic program at UF. As I am expected to know the professional competencies and professional associations in our field to teach current topics and advise students on employment opportunities and more, I have spent several years studying the professionals and professional competencies of those individuals working in the broad field of educational technology. This body of scholarship has been critical to the teaching mission at UF, and keeping our curriculum rigorous and relevant to both those in research and in practice in educational technology. My involvement in this line of research assists me in making curricular and instructional design decisions in my teaching practice, leading to quality discussions during advising and teaching sessions, particularly for our master’s degree students, and mentoring sessions more objective and based on empiricism as opposed to subjective opinions of my own.
As this work has implications for all of those professional and professional associations in our field, it has also resulted in an invitation from the International Board of Standards for Training, Performance and Instruction (ibstpi®) to serve as a scholar of the association to establish international standards for the competencies of professionals in instructional design and beyond. I have used a combination of research methods and approaches to document and validate the standards of professionals in our field by using strategies like rigorous content analysis on job announcements, interviews with diverse professionals in specific contexts in the field, the Delphi method, and classical measurement theory to provide evidence of the validity of the professional competencies and roles of these professionals. My first effort was to document the multimedia competencies of our professionals who are expected to work with a variety of software, resources, and theories to develop robust learning solutions for a wide array of devices and platforms. This eventually led to a broader question of what are the general core competencies of our professionals, including all sorts of job titles like instructional designers, educational software developers, learning designers, educational technologists, project managers, e-learning specialists, and more.
My colleagues and I have even extended this body of scholarship to online instructors by interviewing award-winning faculty from across the United States on their instructional practices and course design efforts, which resulted in a best paper award from the Division of Distance Learning in AECT at the 2019 conference in Las Vegas. This project in particular has been extremely helpful in directing my own online teaching efforts with course design, development, and facilitation. Thus, this body of scholarship has emerged as vital to my teaching, advising, and mentoring experiences by providing empirical evidence and first-hand information about the knowledge, skills, and abilities our graduates need for preparation and success in their future roles. A few of my current doctoral students have even begun to study these competencies for their own research interests, including the project managers in the field of educational technology, simulation directors in healthcare, and instructional designers working in the context of higher education. This body of scholarship has shaped my teaching, advising, mentoring, and overall insights on the importance of studying the competencies of working professionals in our field.
My most recent focus in research binds together my teaching and research into identifying the effective practices and strategies to support all learners in online teaching and learning situations. This body of research seeks to advance the theory, research, and practice in online teaching and learning to optimize the environments for diverse learners, irrespective of background and experiences. Much of this body of scholarship has been in collaboration with some of my closest colleagues, including Dr. Florence Martin from North Carolina State University (NC State) and Dr. Swapna Kumar at the University of Florida. We have addressed a range of issues in online teaching and learning to have an impact in the theory, research and practice of online teaching and learning.
One of our more recent contributions was the creation of the concept and term “bichronous online learning” to describe the systematic and intentional blending of both synchronous and asynchronous online learning strategies and communications to meet the needs of our learners. We started with an extant review of the literature in online learning and have extended this research into systematically investigating the roles, competencies, and practices used by award-winning online educators from universities across the United States and Canada. The term bichronous online learning has been already widely used in the research literature and educational practice in institutions of higher education. We have also developed frameworks for online teaching and learning to assist educators in their own educational practices. Our ultimate goal for this line of research is to continuously improve online teaching and learning practices across the globe.
For my personal and professional learning and development, this research also serves as a way to identify useful practices and strategies for my own educational practices as a college professor. All the ideas and practices we investigate inform our own teaching and learning as reflective educators.