STUDENT ASSESSMENT OF VIRTUAL TEAMS IN ONLINE MANAGEMENT COURSE
Genie Black, Arkansas Tech University
Seventy-four students enrolled in a management web class were asked to assess their perceptions of virtual teams versus traditional face-to-face teams. Each student completed a survey covering demographics, team communication methods, team commitment, team cooperation, team collaboration, overall level of satisfaction with the VT experience, and comparison of their virtual team experience with previous experience in traditional, face-to-face team encounters. Students who perceived virtual teams as more convenient than traditional teams were most likely to give the virtual team a higher rating for effectiveness and to report high levels of satisfaction with their team experience. Students reporting high levels of participation and communication in their virtual team also rated team effectiveness and overall satisfaction significantly higher than did those students reporting low participation and communication levels. Significant relationships were found for perceptions of improved teacher access with team effectiveness and overall satisfaction with the team experience. For traditional team experiences, results from the analysis of variance were significant for team communication (p < .05); significant two-way interactions were found for team collaboration by team communication (p < .05), collaboration by cooperation (p < .05) and team collaboration by team cooperation (p < .05) in a face-to-face team setting. Results for virtual team experiences resulted in significant findings for team communication and team cooperation (p < .05) as they relate to overall satisfaction with the team experience. Implications of the findings are presented.
Business approaches such as empowerng workers, widening spans of control or creating "mini-units" are no longer adequate for obtaining optimal organizational performance. Today, managers are creating teams, networks and boundaryless structures to redesign their organizations in order to better manage change (Peters, 1992). Many firms increase responsiveness by organizing activities around self-managed and self-contained work teams. Such teams are empowered to manage themselves. Their duties include a wide range of tasks such as scheduling; establishing production times, recruiting, hiring, and firing; formulating, tracking and amending their own budgets; making capital investment proposals; handling quality control, inspections, problem solving and trouble shooting; developing and monitoring quantitative standards for quality and production; suggesting and developing prototypes of packaging and new products; and routinely working on other cross-trained teams from sales, product development and marketing (Ziegler, 1997). In team-based work environments, the team is the basic work unit. Team members in the work unit do much of the planning, decision making and implementing necessary to complete their assigned jobs Shonk, 1997).
Organizational development practitioners and managers of organizations have long known the importance of facilitation for successful group processes, but few individuals have tackled the issues of trying to facilitate teams that are connected by distance in time and space. In business and academia alike, working with such teams is increasing tremendously. With the increased relevance of the Internet as a diverse resource in academia, faculty members need to integrate these virtual activities into their current team building strategies as well as learn to continually improve virtual group processes (Lipnack & Stamps, 1997). There has been a rush in academia to integrate the Internet in classroom activities in order to provide new experiences, new environments, virtual teams, communities of practice and enhanced learning. But, like face-to-face teams, meaningful experiences do not happen by chance. Virtual team meetings can be disappointing and frustrating when they lead to information overload, low quality communications or the inability to stay focused on a topic. When a face-to-face meeting is unsuccessful, we tend to review our meeting design as well as our role as facilitator in an attempt to understand what went wrong. When virtual teams’ experiences fail, we tend to blame technology. Instead, it is critical that we extend our understanding about group dynamics in an effort to grasp what happens when individuals interact and collaborate online (Alavi, 1994). The terms "online group", "virtual conference" and "virtual group" refer to many technologies. These may include real-times activities such as video or audio conferencing which link individuals who are in different places participating synchronously; or these technologies may allow individuals to participate from different places at different times in an asynchronous mode.
Different types of media should elicit different types of concerns for teachers. In computer-supported meetings, such issues as how anonymous contributions affect the group and how to test whether or not consensus -- as defined by computer processing of input – is valid must be addressed. In audio conferencing, participants have difficulty knowing who is "present" and facilitators struggle with making sure that everyone with something to say is heard from. Discovery/chat rooms raise the concern of how to facilitate a meaningful conversation about assigned topics among individuals who do not access the information at the same time. Protocols covering such issues as response time, forwarding of messages, who gets copied on e-mail messages, and how the style of electronic messaging influences the way participants feel about the group need to be established for electronic mail. When using computer conferencing, we must consider how to help the participants create the mental image they need for the development of a culture, which will support team processes. Intranets and groupware require that a balance is created between the need for participants to access and process large quantities of information while at the same time developing relationships and qualities like trust (Schrage, 1996). Thus, facilitating virtual teams requires addressing the same issues you face with any team process, but requires that you extend your facilitation to accommodate the virtual environment.
The above concerns, however, are manageable. Whether you are meeting face-to-face or in a virtual environment you can continue to use basic strategies for developing a shared commitment and understanding of the team’s purpose. What is different is the strategy for how the team will remain on track around the stated team goal when members are unable to meet together in a face-to-face learning environment.
Online courses and distance learning have come under attack for a variety of reasons. Misconceptions, such as the belief that virtual learning does not accommodate faculty-student and student-student interactions, are frequently the basis for such attacks. Recently, for example, Business Week ("Wiring the Ivory Tower: But will Online Courses Lower Standards?," 1999) reported that online universities have no sports fields, no dorms and no costly professors! While it may be true that some online courses are poorly developed or may use part-time instructors to grade assignments and answer e-mail, the same can be said of traditional classroom course offerings. These occasional departures from quality, however, should not be used to define all online learning. Rather, the important issues that must be addressed involve how outcomes for virtual courses compare to outcomes for other distance learning approaches and for traditional classroom-based courses; what conditions enhance the likelihood of success for an online course (e.g., what kinds of courses, what kinds of students, etc.); and how web-based courses and other virtual technologies can improve the delivery of material, thus enhancing student learning.
Theory
Three major areas of research lend support for the use of online course technology in general and virtual teams in particular. Educational research provides us with an approach to learning which posits that knowledge must be discovered, rehearsed and verified by the learner. Techniques such as collaborative learning (instructional methods that encourage learners to actively work together on assigned tasks), case study analysis and simulation presented in a virtual environment are consistent with this approach and are believed to successfully develop critical thinking and problem solving skills in students. Furthermore, collaborative learning theory suggests that when students are able to work with peers, rather than working with the instructor or working alone, that the climate for learning is enhanced (Jackson & Winn, 1999).
Media theory (Daft & Lengel, 1986) suggests that the performance of tasks will be enhanced when the requirements of the task are matched to the medium’s ability to convey information. Recent research (Dennis & Valacich, 1999), however, suggests that all media have some degree of inherent richness which can vary in terms of symbol variety, feedback, rehearsability, parallelism and reprocessability. Dennis and Valacich state that no single medium is richest in all of these characteristics and that the relationship between media characteristics and communication processes varies between newly formed versus established groups. This suggests that virtual team strategies may have to be tailored to a specific learning group within the context of the learning environment.
Group interaction theories, such as the process gains and process losses methods of analyzing group meetings (Steiner, 1972; Nunamaker, et. al., 1991), hold relevance for a discussion of virtual team functioning. Research suggests that a team is a complex and complete social system that participates in multiple, independent functions on multiple, concurrent tasks while contained within, and linked to, surrounding systems in the environment. Individual member characteristics, the learning environment, strategies employed, etc. all contribute to a team’s culture and chosen method for accomplishing (or not accomplishing) tasks. Therefore, it is unrealistic to expect that all teams will react in the same manner when engaged in a virtual environment, just as we do not expect identical reactions from teams functioning in a face-to-face environment. Each team, regardless of the work/learning context, has the opportunity to use or not use the tools provided by the environment, heuristic, technology and so forth (Poole & DeSanctis, 1990).
It is reasonable to assume that numerous factors come together in determining team effectiveness and efficiency in group task performance in an online setting. Some of these factors are inherent in the learning process of individuals in traditional, face-to-face team interactions. For example, individual characteristics such as maturity, motivational level, skills and abilities interact with factors involving the type of material being learned, access to the instructor, the type of learning process employed and so forth to impact upon the students’ grades and satisfaction with course/instructor, etc. While these factors are also decisive in virtual team learning, we must also consider issues of technology access, availability and reliability and how the structure of the course itself -- in terms of media used and the extent to which appropriate reward systems are in place to support team learning – interact with team characteristics such as culture.
Method
Seventy-four students enrolled in an online management course participated in this study. All students were assigned to virtual teams that were permanent for the duration of the course. Teams were responsible for collaborating on tasks, conducting brainstorming sessions, completing critical thinking tasks and engaging in problem solving exercises. Team members were also encouraged to use their teammates as resources for course, task and technology questions. All teams were comprised of either three or four members.
A survey was developed to examine how students perceived their virtual team experience. The demographic section of the survey contained eight items: class standing, gender, age, classification (traditional versus nontraditional), number of hours employed in an outside job, expected grade for the course, status of the course (required versus optional), and previous experience in an online course environment. Students were asked to circle the most appropriate response.
Subsequent sections of the survey presented items which were designed to tap into students’ perceptions of team collaboration, team communication, team commitment, and team cooperation. All items were presented in a five-point Likert-type scale. Response formats included satisfaction scales ranging from strongly disagree (1) to strongly agree (5), frequency scales ranging from none (1) to all (5), and a bipolar scale (worse than/ better than). The strongly agree/strongly disagree items focused on student perceptions of course content (e.g., assignments, discussions, etc.), the virtual environment and the virtual team to which they had been assigned. Frequency scale items targeted the extent to which the team participated in activities associated with team collaboration, team communication, team commitment, and team cooperation. The bipolar scale items formed the basis for assessing student perceptions of their virtual team experience in comparison to their previous experience with traditional face-to-face teams. Only select portions of data obtained with this survey have been analyzed for the purpose of this research.
The following hypotheses were examined in this study.
Results
Composite measures were created for the variables of team collaboration, team communication, team commitment, and team cooperation. Inter-item reliability coefficients were calculated for composite measures and Chronbach’s alphas were found to be acceptable (.60 or greater).
Table 2 shows significant correlations for team effectiveness and overall satisfaction with the team experience for virtual and traditional team experiences. Students reported virtual teams as being more convenient than traditional teams. Students also reported higher levels of participation and increased access to the teacher in virtual teams. In addition, students perceived that virtual teams resulted in increased levels of communication and collaboration.
Table 2
Significant Correlations*
Item Team Effectiveness Overall Satisfaction
Virtual Traditional Virtual Traditional
Convenience of team. 64* .33 55* .27
Participation .39* .21 .47 .43
Access to teacher .59* .33 .39 .34
Communication .63* .46 .57* .36
Commitment .37 .30 .29 .30
Collaboration .48* .31 .25 .31
Cooperation .31 .29 .33 .32
To test for differences, four-way analyses of variance (team collaboration, team communication, team commitment, and team cooperation) were conducted for each type of classroom experience (traditional and virtual). The analyses tested for main effects and two-way interactions. Due to empty cells or a singular matrix, higher order interactions were suppressed.
Table 3
Analysis of Variance
Traditional Face-to-Face Team Experience
Variables SS df MS F
Source
Main Effects:
Two Way Interaction:
* Significant F
For traditional team experiences (refer to Table 3), ANOVA results were significant for only one main effect, team communication (p < .05. Significant two-way interactions were found for team collaboration by team communication (p < .05) and team collaboration by team cooperation (p < .05) in a face-to-face team setting.
Table 4
Analysis of Variance
Virtual Team Experience
Variables SS df MS F
Source
Main Effects: `
Two Way Interaction:
ANOVA results for virtual team experiences (refer to Table 4) yielded significant findings for two main effects, team communication and team cooperation (p < .05). In addition, significant two-way interactions resulted for team collaboration by team communication (p < .05), team collaboration by team cooperation (p < .05) and team communication by team cooperation (p < .05). Refer to Table 4.
Discussion
The results of this study lend support for some of the hypotheses that were tested. Students did report increased levels of communication and collaboration (hypothesis 1), as well as increased convenience (hypothesis 2), in their virtual team experiences. From an educator’s perspective, we know that communication, regardless of the medium, must occur in order for learning to take place. Furthermore, educators believe that increased levels of collaborative learning function to enhance individual learning for team members. Thus the virtual team, when properly managed, appears to be at least equal to traditional teams for creating an enhanced learning environment. It is likely that the ease with which teammates could communicate with one another, in conjunction with the novelty of the modes of communication, played a role in improved communication and collaboration. Results of the ANOVA show significant interactions for collaboration with communication and cooperation, and communication with commitment. This suggests that at least some of the variables investigated in this study are intervening/ moderator variables, and warrant further examination.
While correlational data are inconclusive evidence of causal relationships, the findings do provide evidence that the virtual team environment was conducive to increased communication, collaboration and convenience among team members. The methods for communicating in virtual teams provided students with avenues of interaction that may be absent in traditional teams. Students indicated that the use of chat rooms, e-mail and discussion boards; where team members could converse both asynchronously and synchronously; greatly enhanced their level of communication since they were not restricted to meeting only at those times when all members were available to convene at a specified "real" location. In addition, students reported using the virtual communication methods to interact with teammates on a variety of issues, not all of which were even related to team assignments. Thus, virtual communication also appears to have increased the students’ levels of social communication.
In addition, results provide support for the hypothesis that a virtual team environment enhances convenience, participation and access to the teacher. Synchronous communication methods (white boards, for example) enabled students to share web sites, transfer files and communicate in real time. The instructor for the course urged students to meet online at least once each day -- a task that would have been extremely difficult, if not impossible, had students been restricted to face-to-face meetings. Furthermore, the increased level of team member participation appears to have had a positive impact on students’ overall quality of learning, based on student self-reports. Students also indicated that they felt free to "talk" to teammates and the teacher online, where they would be less apt to express an opinion or contradict another team member in a traditional team setting. Finally, access to the teacher was greatly enhanced in the virtual team environment. The typical turn-around in communication between students and instructor ranged from immediate (when all parties were online at the same time) to hourly during working hours. The teacher also communicated with teams evenings and weekends (something that not all faculty might be open to doing). Student in this online course emphasized that they knew more about their teammates and instructor on the first day of class (everyone sent short biographical sketches to one another) than they knew at the end of the semester in traditional settings.
While evidence from the study supports virtual teams as a vehicle for enhanced learning, relationships (correlations) should not be construed as causation. Many of the students enrolled in the virtual management course were first-time web class participants. It is possible that the novelty of the new learning environment was sufficient to result in higher perceptions of team effectiveness and overall satisfaction. Of note, though, is the fact that first-time members of virtual teams were forced to manage not only course material, but also the learning of new technology for interacting with other team members and the instructor. Considering that students were burdened with these additional tasks, one might have expected lower ratings on effectiveness and satisfaction for virtual teams. Furthermore, content analysis of subjective comments presented at the end of the survey suggests that students in virtual teams found the virtual environment to be highly stimulating. They used such descriptors as dynamic and interactive when providing their thoughts about the online resources and ease of exchange of information that was inherent in the virtual setting.
The finding that students in virtual teams reported higher levels of team cooperation than when working in traditional teams must also be viewed with caution. Members of these virtual teams were specifically encouraged to rely on their teammates as a resource for resolving problems and answering questions. Since data on traditional team experiences derives from previous courses in which the students were enrolled, the author has no way of knowing whether or not the students in these traditional team settings were urged to interact for such purposes.
Mention also needs to be made about the course content and structure, and the potential impact it had on students’ perceptions in this study. This management course had been delivered online for three semesters at the time of this experiment. Over time, the course content has been modified and the course structure has been changed in order to improve both content and delivery of material to be learned. Assignments have shifted from individual tasks that merely required the students to use online resources to team tasks that demand high levels of collaboration and interaction among team members. To facilitate collaboration, students use chat rooms and discussion boards, as well as e-mail. In addition, course content has been supplemented with audio and video presentations and an interactive online text that students really enjoy. In comparison, the majority of web-based courses offered to these students have not been developed to this extent. Some courses only require that the student simply access an online document that instructs them to read certain pages from their text. As a result, students with previous web class experience may have found the methods of content presentation in this course to be novel, which may, in turn, have resulted in inflated perceptions of effectiveness and overall satisfaction. Without question, student perceptions have been potentially confounded by a variety of variables that were not controlled for in this study.
A final word of caution pertains to the students themselves who comprised the virtual teams. Not all teams, and not all students, were successful in this course. Several students self-selected out of this course within the first two weeks of the semester, and two students failed the course. In addition, since data for this study were collected at the end of the semester, teams that were not particularly successful at the beginning of the semester had ample opportunity to learn from their mistakes over time. Considerable effort went into coaching teams that were floundering earlier in the term. Without this intervention, it is possible that student perceptions of team effectiveness and overall satisfaction with the team may have been significantly lowered. In addition, most students who were performing very poorly self-selected out of the course before data were collected. Thus, students/teams who participated in this study were, for the most part, successful in the course. Further investigation using more rigorous experimental techniques is certainly warranted in order to control for these variables.
Conclusions
Creating successful virtual teams in academia differs from merely assigning individuals to join together in problem resolution. The virtual team requires additional support from both the faculty member and the institution. From an institutional perspective, it is requisite that the administrators provide the necessary technology needed to make virtual environments a success. Merely putting the technology in place is insufficient, as a successful virtual learning environment requires a tremendous amount of coordination. This is where technology support groups become critical. Technological support teams play a vital role in the functioning of virtual learning environments. Sufficient personnel must be available to upgrade and maintain technology as needed, and the technology employed must be user friendly. Web masters must be proficient at their jobs and function to facilitate faculty efforts rather than to impede progress by placing constraints that limit teaching innovation. In addition, reward systems should be in place to encourage faculty excellence in course structure and presentation. Redesigning an old course, or designing a new one, to fully integrate today’s expansive technology is no easy task. Support for such endeavors should include mentoring programs, rewards for performance and innovation in teaching, release time for course development, faculty development (e.g., attending symposiums on creating successful courses) and encouragement from department heads and deans.
From the faculty perspective there are a number of considerations that should go into development of virtual teams. Faculty must take care to clearly express the purpose for the team and identify/clarify team goals as they pertain to individual tasks and overall expectations. An effort must be made to create a positive team atmosphere at the outset. Teachers can greatly improve team functioning by encouraging a strong team culture (e.g., create assignments that help members to identify with one another (e.g., biographical sketches); and setting an appropriate tone for virtual communications (e.g., use specific message formats, e-mail protocols). Polling student opinions on a topic or presenting material in a different format can enrich team communication. Feedback for the teams is essential. Good team contributors can be thanked by private e-mail or recognized in class announcements. Poor team contributors may require additional support in order to communicate effectively with the team and instructor. Individuals who stay within the role of "reader" can be thanked for being present for team conferences. Active teams can be recognized. Slow teams may need to have the pace of the work adjusted to some extent. Training teams to summarize and synthesize multiple response before sharing them with the class helps others to understand issues or creates an awareness that more information is needed. Tracking participation in teams can highlight who is doing what and clarify ways of improving team functioning.
Such efforts take time and practice on the part of the teacher. There is no single right approach to virtual team management. The key is to identify and use information on what is occurring in the teams in order to become a better facilitator. If this sounds familiar, it may be because facilitating a virtual team requires the same expertise that is needed for facilitating face-to-face teams – but the work must be coordinated through technology.
Clearly, there are risks and special concerns involved with virtual teams. The rewards, however, can be great. When properly designed and given appropriate interdependent tasks, virtual teams may provide more innovative solutions to problems and create new ways for students to learn and interact in an educational setting.
References
Alavi, M. (1994). "Computer-mediated collaborative learning: An empirical evaluation," MIS Quarterly, 18 (2), 150-174.
Daft, R. & Lengel, R. (1986). "Organizational information requirements, media richness, and structural design," Management Science, 32(5), 554-571.
Jackson, R. & Winn, W. (1999). Proceedings of the Computer Support for Collaborative Learning Conference.
Lipnack, J. & Stamps, J. (1997). Virtual Teams: Reaching Across Space, Time and Organizations with Technology. John Wiley and Sons.
Nunemaker, J., Dennis, A., Valacich, J., Vogel, D., & George, J. (1991). "Electronic meeting systems to support group work," Communications of the ACM, 34(7), 41-61.
Peters, T. (1992). Thriving on Chaos, 256. New York: Alfred Knopf.
Poole, M. & DeSanctis, G. (1990). "Understanding the use of group decision support sysems: The theory of adaptive structuration" in Fulk, J. & Steinfield, C. (Eds.), Organizations and Communication Theory, Sage, Newbury Park, CA.
Schrage, M. (1996). No More Teams: Mastering the Dynamics of Creative Collaboration. Doubleday.
Shonk, J. (1997). Team Based Organizations. Chicago: Irwin.
Steiner, I. (1972). Group Processes and Productivity, New York: Academic Press.
Ziegler, B. (1997). "Gerstner’s IBM Revival: Impressive Incomplete," Wall Street Journal, 27, 81 - 84.