Implementing Six Sigma in a Distance Learning Application

—This paper presents the results of a six sigma study that targets the issue of measuring the current performance of a new online delivery system that is offered by the College of Engineering and Computer Science (CECS) at the University of Central Florida (UCF) and assessing its capabilities. The objective of the study is to improve the faculty and students’ satisfaction and ensure a leaner process in delivering online courses. The results and recommendations of the study are also presented.


I. INTRODUCTION
The trend towards providing distance learning opportunities has been enormously increasing over the last few years, especially in academic institutions. This trend was prompted by the introduction of technological advances that made learning materials accessible by students and instructors from any location at different times and with different modes. Distance learning has different modes that can be applied according to the institution's needs synchronously (all parts communicate at the same time) or asynchronously (delay in interact with each other while they are in different places) [1]. According to research conducted by the Sloan Consortium, distance learning appears to be a vibrant part of higher education, with 83% of higher education institutions offering some form of distance learning [2]. Researchers examining distance learning programs at higher education institutions report many cases of successful, well-developed and thriving programs (e.g. Moore, 2004) and others that stagnate, shrink or have been discontinued (e.g. Garrett, 2004;Schell, 2004). For many of the latter institutions "decisions about distance education are made too often without adequately considering the broader institutional context" [3] and "some institutions that are struggling to keep up with the demand for internet-based courses have made a conscious decision to serve students immediately and plan later" [4].
One reason for the lack of success of many of the online programs and similar innovations is that they have never been fully institutionalized within their organizations [5]. In other words, they have not become a "normal" and integral part of the institution, losing their "special project" status [6].
The College of Engineering and Computer Science at UCF has been a leading institution in distance learning for the last 25 years. The college started the distance education experience by using video tapes and closedcircuit Television (CCTV) in order to expand the distance learning experience all over the United States and abroad.
In the spring semester of 2007, the College of Engineering and Computer Science acquired TEGRITY; it is the first student achievement system that impacts learning across the entire institution, improving retention and student satisfaction. TEGRITY makes class time available all the time by automatically capturing, storing and indexing every class on campus for replay by every student. Educators know that the more students can see, hear, and experience class resources, the better they learn. With TEGRITY students quickly recall key moments or replay entire classes online, on their iPods and cell phones [7]. TEGRITY has been since used for all Florida Engineering Education Delivery System (FFEDS) courses as a new system that enhances the overall quality and learning experience for both the students and faculty. TEGRITY allows instructors to do more than just capture, store, and index class content, and give students more control over their learning experience and outcomes. In less than two years, TEGRITY use has exploded across main campus and remote locations. Online enrollment is up and college leaders are excited by the innovative ways faculty are using TEGRITY to enrich the student experience and teach more effectively.
This study was initiated and conducted by the Department of Industrial Engineering and Management Systems and the Center for Online and Virtual Education (COVE) at UCF. A six sigma study was implemented with the objective of providing a practical measurement tool to assess the performance of the new system in order to improve customer (student and faculty) satisfaction and assure a leaner process in delivering online courses.
II. SIX SIGMA IMPLEMENTATION The old FEEDS and the new delivery system (TEGRITY) were running in parallel since the spring of 2007. COVE has received several complaints from faculty and students at the same time about transitioning from the old legacy system to TEGRITY. COVE recognized the need to initiate a study to assess the current situation in the college of engineering and computer science at UCF.
It was decided to implement a Six Sigma DMAIC (Define, Measure, Analyze, Improve and Control) methodology to assess and measure the performance of its online delivery system with the objective of increasing the satisfaction of faculty and students in the college.
Six sigma is one of the more recent quality improvement initiatives to gain popularity and acceptance in many industries across the globe. Its popularity has grown as the companies that have adopted Six Sigma claim that it focuses on increasing the wealth of the shareholders by improving bottom-line results and achieving high quality products/services and processes. Thus, it is claimed that the implementation of Six Sigma brings more favorable results to companies in comparison with traditional quality initiatives in terms of turning quality improvement programs into profits. Success stories of big corporations that have adopted Six Sigma, such as Motorola and General Electric (GE), have been reported in various papers [8].
The core of the six sigma quality initiative is the DMAIC (Define-Measure-Analyze-Improve, Control) cycle. It is an integral part of Six Sigma that provides a useful framework for conducting Six Sigma projects [9]. The following sections explain how the DMAIC cycle was implemented to the project.

A. Define -What is the opportunity for improvement?
In the define phase, we focused on understanding the process of the online course delivery system at UCF and the need for transitioning from the old to the new system and listening to the voice of the customers, represented by faculty and students. While aiming at improving the current online delivery system, the following objectives were identified: 1) Promote engineering education at UCF and at the state of Florida. 2) Achieve the university mission of expanding the current online offerings in the next few years to include the delivery of complete online programs. 3) Increase the satisfaction of both students and faculty. 4) Eliminate the need for scheduling an entire classroom for pre-recording. 5) Improve the quality of the old online delivery system.
As part of the six sigma define phase, we focused on mapping the old process and identifying the existing problems associated with the old system. The structure of the old system is shown in Figure 1.
The university implemented this delivery system 25 years ago by having six multi-media rooms at maximum capacity used for lecture recording and a large video encoding control room. This system has been enhanced to edit NTSC 640x480 videos to be saved as video tapes.
About 6 years ago, the cable was taken out of the back of the VCR and was inserted into a video capture card in an encoding machine; the information was encoded and put on a website where students can watch and download recordings. This 640x480 quality becomes worse with 25 years old equipment and cabling, it turned to 320x420 pixels. The resulting quality of resolution was acceptable at that time, but the need for higher resolution and better quality could not be overlooked.
The University of Central Florida aims to expand the current online offerings in the next few years to include the offering of complete online programs and, at the same time, enhancing the face-to-face course delivery. The only way to expand the online offerings using the FEEDS system is to create more studio space which will be typically staffed with full-time personnel to maintain, monitor and/or adjust in real-time the various input/output systems, in addition to the outfitted studios with expensive devices such as video cameras, document cameras, etc.
The Center for Online and Virtual Education (COVE) at the university addressed the need for a change in the way of capturing and delivering the contents of courses. This was done by reducing the hardware requirement, improving the software as much as possible and enabling each instructor to have his/her own mini-studio which allows them to record lectures on their own without being physically present in a specific classroom. Instructors can record lectures using their own computers or laptops from any room in the building or anywhere outside the university. Figure 2 displays the new model which enabled the engineering departments to avoid conflicts and other issues of scheduling courses. Most of the classrooms on campus have projectors and computers with additional web cameras and microphones that enable the instructors to easily record their lectures and deliver them to students.
To identify the characteristics that can lead to a higher quality of online delivery system of courses, listening to the voice of the customer was essential. To achieve this, several meetings, interviews were conducted with faculty,  Figure 3 shows the Critical to Quality (CTQ) Tree, starting by identifying the need, drivers, and all of the critical-toquality characteristics (CTQs) that have been identified [8].

B. Measure -Collecting Data
We used several data collection tools, such as surveys, interviews, focus groups, observations and complaint analysis. A data collection plan was created and two online surveys were conducted to gather responses from faculty and students about the old and the new online course delivery systems and to identify opportunities for improvement. The surveys targeted both faculty and students who used both systems.

1) Student Survey
The student survey was distributed through COVE website where the students view their online lectures. It was available for the last two weeks of the 2007 fall semester and targeted all engineering students who used both systems, old FEEDS and TEGRITY. The survey was structured to collect data and information in the following major areas: 1) How often do the students watch their classes online? 2) How often do the students use the old FEEDS system in comparison to TEGRITY? 3) How much relevant are the features provided by TEGRITY to the students who prefer watching their classes online? 4) How much important the quality of resolution of an online lecture, especially when the instructors use a certain software in class?

5) Does the College of Engineering and Computer
Science move in the right direction with online lectures by utilizing the TEGRITY system?
The survey consisted of both closed (multiple choice items) and open-ended questions. There were 26 multiple choice questions that were structured in a certain logic to correctly identify the desired objective of assessment in addition to two open-ended questions that provide students to enter their own comments about other features that they liked most about TEGRITY and any recommendations for improvements in the new online course delivery system. The survey allowed the classification of students by their standing, undergraduates or graduates, degree sought, masters or Ph.D. The survey also gathered details about: • The type of platform students use (PC, MAC or Linux) and the different problems encountered while using different platforms • How often does the student use the old FEEDS or TEGRITY • Classification of students who prefer using FEEDS and never used TEGRITY. Such data was used to identify other problems with FEEDS and reasons for not using TEGRITY

2) Faculty Survey
The faculty survey was sent via e-mail to all engineering faculty members and was available for the last two weeks of the 2007 fall semester. The survey consisted of both closed (multiple choice items) and openended questions. Instructors who never used TEGRITY to record their lectures were directed to the end of the survey for specific extraction of other data. The faculty survey was structured to collect feedback from faculty in the following areas: 1) How much do the recording features provided by TEGRITY increase their teaching effectiveness? 2) What do faculty members like about recording a class anywhere and at anytime instead of being physically present in a classroom?

3) Does the College of Engineering and Computer
Science move in the right direction with online lectures by utilizing the TEGRITY?
The surveys allowed the collection of a real feedback from students and faculty about the two online course delivery systems, identifying issues and problems, and

C. Analyze
The populations of faculty and students surveyed were 100 and 2300 respectively. A 36% response rate was realized from faculty and 40% response rate from students. This indicates the validity of the results that would be used to initiate plans for improvement.

1) Student Survey Results
The collected data showed the following information: • 53.3% of the students who responded were undergraduates and 46.7% were graduate students, of which 61.85% were master's students and 38.2% were Ph.D. students. These results are displayed in the bar chart shown in Figure 4 and Figure 5. It was also an interesting observation to know that 91.4 % of the engineering students watch their classes online while 8.6% only of the engineering students online prefer watching a class in a classroom. Figure 6 represents the percentage of students who use both the old FEEDS and TEGRITY systems to view their classes online. Results show that 46.5% of the engineering students never used the Old FEEDS system to watch their classes online. On the other hand 3.3% only of the students never used the TEGRITY system to watch their classes online. However; some students, use both systems to view their classes online and this might be due to certain problems in either one of the systems that will let them navigate between the old FFEDS system and the TEGRITY system. These problems were identified by students' open-ended questions of the survey.
The collected data were used to identify the factors that influence the quality of the new online course delivery system (TEGRITY) and summarized in the cause and effect diagram in Figure 7.
The major advantages that students cited for the TEGRITY system over the old FEEDS system were identified as follows: 1) The ability to watch the instructor while he/she is teaching a class using the Picture-in-Picture feature of the TEGRITY. 2) The ability to easily review and learn the material presented through the search feature in TEGRITY, watch the lecture in a slow or fast speed, and view all slides in one window. 3) An improved resolution quality of presentations, such as PowerPoint slides, Adobe PDF presentations and other software. 4) The ability to transform the notes written by the instructor using the TEGRITY digital pen into printable slides that can be later printed by the student.
While the students believe that the TEGRITY represents an improvement over the Old FEEDS system in delivering online courses, they still think that it needs more improvement. These opportunities for improvements were identified as follows: 1) The ability to view the lectures in a LIVE mode although results show that 63.3% of the students never used this feature that is available with the Old FEEDS system. 2) The ability to have live chat with instructors while they are teaching a class.
3) The TEGRITY system is till not as reliable as the Old FEEDS system; sometimes the system freezes while the instructor uses the TEGRITY Digital Pen; this might be due to hardware incompatibility. 4) Incompatibility with internet browsers other than the internet explorer.
The survey also highlighted more improvement opportunities for the classroom environment such as having more microphones that capture the students' questions or discussions; which allows the students who watch the class online to listen clearly to any discussion held in the classroom in addition to changing the position of the instructor camera in classroom since the current camera takes a profile picture of the instructor; students prefer having a camera that faces the instructor while he/she is teaching in a classroom.

2) Faculty Survey Results
Survey results indicate that the percentage of faculty who use TEGRITY to record their classes is 80%. On the other hand, the instructors liked the ability to record a class at a place other than the classroom and without being physically present in a classroom to teach an online class.
Results also indicate that some instructors are not aware of this feature when recording. This creates a need for more training on the features provided by the new system. Instructors believe that the improvement in quality and resolution is so important for them and for their students. Opportunities for improvement in the current system were identified by the instructors as follows: 1) Create an environment similar to a classroom for students who view classes online by including live broadcast and chat with the students. 2) Better and faster display of written notes when using the digital pen. 3) Smart whiteboards will increase the teaching effectiveness.
Survey results, shown in Figure 10, indicate that both students and faculty think that the College of Engineering and Computer Science is moving in the right direction with online lectures by utilizing the new TEGRITY system.

D. Creating and Controlling Improvement
Opportunities In the last phases of the DMAIC methodology, it was essential to test the degree of satisfaction of students and faculty of the new system. Data were gathered from responses to the following statement: The new TEGRITY system is better than the Old FEEDS system. The following table summarizes the results. In testing the hypothesis that both faculty and students have the Dialasame degree of satisfaction from the improvements made by introducing a new distancelearning delivery system, the following summary obtained based on a Z-test. At the 95% confidence level, the tabulated value of 1.96 for the test statistic, Z, was greater than its calculated value, indicating that there is sufficient evidence that there is no difference in the degree of satisfaction of faculty and students from the introduction of the new TEGRITY system [10].
Further analysis of the data resulted in creating several other opportunities for improvement as follows: 1) Provide training sessions prior to the start of the semester on the new system for instructors who are not comfortable with the transition from the old system to the new one or for others who want to learn more about the capabilities of the TEGRITY system to increase their teaching effectiveness. 2) Provide internet videos on "How to Use TEGRITY" on the COVE website or within each WebCT course and notifying students and faculty about the video link. 3) Provide a telephone support that help students or faculty in fixing problems that are encountered while they are not on campus. 4) Provide Smart Whiteboards in the classrooms. 5) Increase the number of microphones in the classroom so as to capture students' discussions and questions or replace the existing microphone with ones that better capture students' discussion and communications with the instructors.
In order to further enhance the performance of the online delivery system in the College of Engineering and Computer Science at UCF, the process should be standardized in order to ensure that gains from improvement are documented, which create more room for further and better measure of satisfaction.

III. CONCLUSION
The DMAIC methodology was used to assess the performance of the current online delivery system at the College of Engineering and Computer Science at UCF. The results showed that the College of Engineering and Computer Science has successfully modernized the existing distance learning delivery system. The improvements realized from the new system and further attainable gains will allow UCF to expand and scale the current online offerings to achieve a goal of 200% increase by 2009. The mini-studio model allows instructors to create educational contents without being physically present in the classroom. The Center for Online and Virtual Education at the college believes that this will significantly impact the potential achievement of UCF students and will ultimately further the economic growth of the State of Florida.