Yesterday I drew a line under the initial phase of the data collection when I closed the online questionnaires to further responses.
To summarise the target students for this research:
705 students are registered to study the year 1 course in 2008, and thus were invited to complete the Pre-Course questionnaire. 567 responses were returned to the questionnaire which yielded 506 usable sets of data once duplicates were removed. As a principle the latest version of the questionnaire has been included unless this was a blank response, in which case the next earliest that included answers was taken. Overall this equates to a 72% response rate to the Pre-Course Questionnaire.
470 students are registered to take the Information Management and Control module which the initial phase has focused on. Of these 74 students were retaking the module after failing or deferring it in a previous year. 381 students submitted a copy of the diary that they kept for week 2 of the module, of which 363 were usable for research purposes. This equates to a response rate of 77% for the diary.
In respect of the module assessment 447 students submitted their first assignment and 438 submitted assignment 2, a submission rate of 95% and 93% respectively. In terms of module outcome this results in 430 students passing the module - a pass rate of 91%.
348 students submitted a response to the Post-Module questionnaire. After duplicates were removed (following the same criteria as for the Pre-Course questionnaire) this yielded 310 usable responses - a response rate of 66%.
When these various sets of data are combined, the result is a total of 221 complete sets of data that include responses to the two questionnaires, submission of the diary as well as details of student's assessment marks and their participation in the VLE forums. This equates to 47% of the module population, however, as only 2 students retaking the module submitted a complete set of data this is effectively 55% of new students taking the module.
In respect of profile of the full sets collected, 56% are from male students (compared with 65% across all students taking the module), which would appear to be representative. What appears less representative is that 99% of the students submitting complete sets of data are passing the module whereas only 1% (2 students) have failed it. This does suggest that a proportion of students (up to 45) may already have dropped out or deferred before the module ended. What is not clear is the extent that time may have caused, or contributed, to this.
The next phase is to analyse the data. All 221 sets of data are now printed and a visual examination will be the first step. This should reveal those factors which seem to contribute to time pressures, and so will be extracted into SPSS for more detailed analysis.
Never a dull day!!
Monday, 21 April 2008
Sunday, 6 April 2008
Data Collected?
Maybe this scenario is familiar to you. You have a period of free time coming up and you plan to get a lot of work / research / writing done - then life intervenes!! This was the story of my Easter break. I had some leave to use up before the end of March so booked to take the week off after Easter. My plan was to sort out the data and get it into a form ready for SPSS. All went well until Easter Sunday and then the dreaded 'flu virus struck and that was me finished for a whole week! It is a long time since I've been so badly affected to the point that I have not even wanted to switch the computer on!!
So now I am trying to catch up with my schedule. In reality I am still collecting data from the students. Despite reminders there are still a number who have not sent me their diary, however, I reckon this reflects the accuracy (or lack of it) so I am no longer pushing for these. Actually I have done pretty well and have 358 diaries to work with.
I am still pressing students to complete the post-module questionnaire as the response has not been as good as I had hoped for. 327 have so far completed it which is not bad but leaves quite a few incomplete sets of data (pre-course questionnaire, diary, post-course questionnaire) which if I can get a further 80 or so to submit will be good.
I have now finished analysing the module forums which give me actual details of messages posted and replied to, and at what time on which day these were made. Combined with the Blackboard stats, this not only allows me to compare the actual access and activity against the diaries but will also give me a picture of whether a student was posting to the study schedule and, if not, how far behind they may have got.
The last set of data that I have yet to complete is the assignment mark data. This is now waiting for one or two slow tutors to return their assignment 2 results. Once in I should be in a position to start printing out sets of results per student. As of today I have around 190 sets of complete data but with luck (and the cooperation of the students) I am hoping to get this up to around 240 students ie half the students taking the module.
The last three months have been very pressurised in terms of data collection. I am now looking forward to a period of analysis and drawing interim conclusions from the data.
So now I am trying to catch up with my schedule. In reality I am still collecting data from the students. Despite reminders there are still a number who have not sent me their diary, however, I reckon this reflects the accuracy (or lack of it) so I am no longer pushing for these. Actually I have done pretty well and have 358 diaries to work with.
I am still pressing students to complete the post-module questionnaire as the response has not been as good as I had hoped for. 327 have so far completed it which is not bad but leaves quite a few incomplete sets of data (pre-course questionnaire, diary, post-course questionnaire) which if I can get a further 80 or so to submit will be good.
I have now finished analysing the module forums which give me actual details of messages posted and replied to, and at what time on which day these were made. Combined with the Blackboard stats, this not only allows me to compare the actual access and activity against the diaries but will also give me a picture of whether a student was posting to the study schedule and, if not, how far behind they may have got.
The last set of data that I have yet to complete is the assignment mark data. This is now waiting for one or two slow tutors to return their assignment 2 results. Once in I should be in a position to start printing out sets of results per student. As of today I have around 190 sets of complete data but with luck (and the cooperation of the students) I am hoping to get this up to around 240 students ie half the students taking the module.
The last three months have been very pressurised in terms of data collection. I am now looking forward to a period of analysis and drawing interim conclusions from the data.
Saturday, 15 March 2008
.... and Even More Data
As predicted this week has seen a deluge of diaries hit my inbox. Overnight from Sunday to Monday 75 arrived and this morning I have 326 diaries in total, hopefully with more to come.
I have now got opening the diary file, removing the password protection, copying and pasting the data into the master file and then saving the original diary down to a fine art. I reckon each now takes me about 2 minutes - so getting on for 3 hours work just to collect the data.
The post-module questionnaire went live last Sunday and already 200 students have completed it. All being well I am anticipating having a complete set of data (questionnaires and diary) for around 300 students. More would be nice!
One of the benefits of being in the UK is the ability to pop into IOE. I was in London for a workshop on advanced scholarship 10 days ago and arranged to have 20 minutes with my supervisor on the way home. It really is helpful to use occassions like that to take stock and to articulate where you are at with someone. Makes me think that a 'buddy' system could work equally well for times in between.
Incidentally it was graduation day at IOE so nice to see the students dressed up in all their finery and caps and gowns. Have to say the EdD robes look well worth the time and effort:-)
I have now got opening the diary file, removing the password protection, copying and pasting the data into the master file and then saving the original diary down to a fine art. I reckon each now takes me about 2 minutes - so getting on for 3 hours work just to collect the data.
The post-module questionnaire went live last Sunday and already 200 students have completed it. All being well I am anticipating having a complete set of data (questionnaires and diary) for around 300 students. More would be nice!
One of the benefits of being in the UK is the ability to pop into IOE. I was in London for a workshop on advanced scholarship 10 days ago and arranged to have 20 minutes with my supervisor on the way home. It really is helpful to use occassions like that to take stock and to articulate where you are at with someone. Makes me think that a 'buddy' system could work equally well for times in between.
Incidentally it was graduation day at IOE so nice to see the students dressed up in all their finery and caps and gowns. Have to say the EdD robes look well worth the time and effort:-)
Saturday, 1 March 2008
Data, Data and More Data!
Funny how time disappears so quickly!! Over the last couple of weeks I've been processing the diaries as they come in. I now have over 160 sets of data but the real rush will come in a weeks time when students submit the assignment that the diary is attached to. My guess is that most are waiting until after they have finished writing the assignment before they send the diary to me. 479 students are taking the Information Management module so I've probably only received around a third of the diaries so far!
One tip to pass on from the experience with the diaries is to write down safely any passwords used. I password protected the diary spreadsheets leaving just the cells requiring entry to be accessible. The spreadsheet collected all the export info into a single hidden column of 1133 items so that the cut and paste into the analysis spreadsheet would be straightforward. Only when I went to unprotect the first diary did I discover that the password I thought I'd used did not work and remissly I had not written it down anywhere. You can imagine the frustration trying different words to access the data without success. Fortunately a search on the web revealed a handy bit of software that can be loaded and will unprotect spreadsheets. I don't know how it works but it does and now I have my data - relief :-).
As indicated I am collecting the diary results into an analysis spreadsheet which allows me to analyse each line of data to get summaries and averages. It also allows me to build a proportional chart from all the responses so that I can compare an individual student with the 'average'. From this comparison I am hoping to identify like groups of students who have similar time profiles, and who can then be followed up in greater detail.
The other thing I've been doing since the last post is to analyse the students' contributions to the online discussion forums. A couple of years ago I wrote a spreadsheet that will produce a simple analysis of messages and tells me who has posted with date and time, whether the post was an original message or a reply to another participant and how many times the message was read. From this I will be able to verify the online (VLE) entries in students' diaries and gives a more accurate profile than the statistics of hits that Blackboard provides. It is a bit lengthy with 24 tutor groups to open and then to collect, copy and paste the messages from each of the 10 forums but it does the job.
Gradually all the different sources of data are being combined into summary pages for each student. Once this is complete I will have a better idea of the full set of variables available for detailed analysis. In the meantime my next job is to prepare the post-module questionnaire ready for the 10th March when students submit the end assignment. No rest for the wicked as they say!!
One tip to pass on from the experience with the diaries is to write down safely any passwords used. I password protected the diary spreadsheets leaving just the cells requiring entry to be accessible. The spreadsheet collected all the export info into a single hidden column of 1133 items so that the cut and paste into the analysis spreadsheet would be straightforward. Only when I went to unprotect the first diary did I discover that the password I thought I'd used did not work and remissly I had not written it down anywhere. You can imagine the frustration trying different words to access the data without success. Fortunately a search on the web revealed a handy bit of software that can be loaded and will unprotect spreadsheets. I don't know how it works but it does and now I have my data - relief :-).
As indicated I am collecting the diary results into an analysis spreadsheet which allows me to analyse each line of data to get summaries and averages. It also allows me to build a proportional chart from all the responses so that I can compare an individual student with the 'average'. From this comparison I am hoping to identify like groups of students who have similar time profiles, and who can then be followed up in greater detail.
The other thing I've been doing since the last post is to analyse the students' contributions to the online discussion forums. A couple of years ago I wrote a spreadsheet that will produce a simple analysis of messages and tells me who has posted with date and time, whether the post was an original message or a reply to another participant and how many times the message was read. From this I will be able to verify the online (VLE) entries in students' diaries and gives a more accurate profile than the statistics of hits that Blackboard provides. It is a bit lengthy with 24 tutor groups to open and then to collect, copy and paste the messages from each of the 10 forums but it does the job.
Gradually all the different sources of data are being combined into summary pages for each student. Once this is complete I will have a better idea of the full set of variables available for detailed analysis. In the meantime my next job is to prepare the post-module questionnaire ready for the 10th March when students submit the end assignment. No rest for the wicked as they say!!
Sunday, 17 February 2008
Getting Down to Stats
This last week has been one of facing up to the volume of data that this research is generating.
Completion of the pre-course questionnaire continues at a trickle with 561 students having now completed it yielding 481 usable sets of data. So far 70 students have returned their completed diaries. Interestingly some appear to have chosen to give someone else's ID for the diary!
How to analyse the diary content has been high in my thoughts. During the teaching week in London last week I had a one-to-one session with one of the lecturers who knows about stats and SPSS. It was really helpful to 'voice' the research and data issues, and the result is I am much clearer about the form in which the data needs to be recorded .
One issue that stats cannot resolve is that of sequencing ie the order and times that students spend on different activities during a day. My thinking at the moment is that this will have to be looked at visually. I have therefore created a spreadsheet that shows the time students spend on the four main activities (resting, working, studying and other) as colour coded 10 minute blocks. Already this serves to show whether an individual is blocking longer periods of time or grabbing short sessions, and whether this is happening during the week or at weekends.
Another tool I have created is a proportional 2-D chart that summarises the student's week, and is based on one illustrated by Gershuny (see previous blog). Created in Excel, it shows the proportion of time spent on the 4 main activities as a grid with columns of width equal to the percentage of time spent on that group of activities. Each column then sub-divides into the time activities within that group with the height of each block proportional to the time spent on it. One idea for using this is to print each chart on acetate and to overlay them. This should help to identify common patterns of time usage (and may be simpler than trying to do multi-variate analysis!!). I'll let you know how I get on.
I'm one of those people who is more than happy crunching number but still get lost with the more advanced stats - mainly due to not using the techniques often enough. For anyone in a similar position, or even for those who are unsure about statistics, can I recommend a book by Reva Berman Brown and Mark Saunders called Dealing with Statistics. The great thing is that there are no formulae or numbers included at all!! What they do is to talk you through the basics of statistics and what techniques can / should be used for different purposes. It is only a short book so doesn't take long to read but quickly gives the sorts of reminders that a rusty mind like mine needs.
Completion of the pre-course questionnaire continues at a trickle with 561 students having now completed it yielding 481 usable sets of data. So far 70 students have returned their completed diaries. Interestingly some appear to have chosen to give someone else's ID for the diary!
How to analyse the diary content has been high in my thoughts. During the teaching week in London last week I had a one-to-one session with one of the lecturers who knows about stats and SPSS. It was really helpful to 'voice' the research and data issues, and the result is I am much clearer about the form in which the data needs to be recorded .
One issue that stats cannot resolve is that of sequencing ie the order and times that students spend on different activities during a day. My thinking at the moment is that this will have to be looked at visually. I have therefore created a spreadsheet that shows the time students spend on the four main activities (resting, working, studying and other) as colour coded 10 minute blocks. Already this serves to show whether an individual is blocking longer periods of time or grabbing short sessions, and whether this is happening during the week or at weekends.
Another tool I have created is a proportional 2-D chart that summarises the student's week, and is based on one illustrated by Gershuny (see previous blog). Created in Excel, it shows the proportion of time spent on the 4 main activities as a grid with columns of width equal to the percentage of time spent on that group of activities. Each column then sub-divides into the time activities within that group with the height of each block proportional to the time spent on it. One idea for using this is to print each chart on acetate and to overlay them. This should help to identify common patterns of time usage (and may be simpler than trying to do multi-variate analysis!!). I'll let you know how I get on.
I'm one of those people who is more than happy crunching number but still get lost with the more advanced stats - mainly due to not using the techniques often enough. For anyone in a similar position, or even for those who are unsure about statistics, can I recommend a book by Reva Berman Brown and Mark Saunders called Dealing with Statistics. The great thing is that there are no formulae or numbers included at all!! What they do is to talk you through the basics of statistics and what techniques can / should be used for different purposes. It is only a short book so doesn't take long to read but quickly gives the sorts of reminders that a rusty mind like mine needs.
Sunday, 10 February 2008
Diary Recording Week
The official notification that the research proposal has been approved by IOE came through this week - nice to see it in writing and signed off by the chair of the committee.
This week the students have (hopefully) been recording the time they spend on the different time activities. The week has run from midnight last Sunday and finishes at midnight tonight. Fortunately there have not been too many queries about which code to use. Either this means the instructions were very clear or that they have not actually filled in the form yet!! We will see next week how many start to email their completed spreadsheets.
Yesterday was a face to face day for the students targeted for this research. Around 150 students came to Reading for a day of talks covering subjects and specific aspects of the course that they need to be aware of. It gave me the opportunity to introduce and explain the project and to field questions about the research and particularly issues involving the 7 day diary. Most students seem to be clear about what is involved which is a blessing.
When I decided to set the diary for this week I overlooked the fact that students would be coming to Reading, however, on reflection that will be no bad thing as it will allow me to look at the time impact on students of taking a day out of their weekend. Certainly compared to face to face days arranged during the week the numbers attending were lower.
The discussion forum for the diary has mainly been used to discuss the value of podcasts by students. Many point to the value of having audio versions of the printed materials either for downloading or as CDs which they can play in the car or train while travelling to work or between client sites. However, they also want the printed version too!
There is still a slow trickle of students completing the pre-course questionnaire. 553 returns have been made but this reduces to 472 after duplicates and substantially incomplete forms have been removed.
In the time I've had this week I've started to clean the pre-course questionnaire data for entry into an Access database. I've also collected the base details of all the current students from the College's student records system. Once I've learned better how to work in Access I should be able to start building a comprehensive set of records!
This week the students have (hopefully) been recording the time they spend on the different time activities. The week has run from midnight last Sunday and finishes at midnight tonight. Fortunately there have not been too many queries about which code to use. Either this means the instructions were very clear or that they have not actually filled in the form yet!! We will see next week how many start to email their completed spreadsheets.
Yesterday was a face to face day for the students targeted for this research. Around 150 students came to Reading for a day of talks covering subjects and specific aspects of the course that they need to be aware of. It gave me the opportunity to introduce and explain the project and to field questions about the research and particularly issues involving the 7 day diary. Most students seem to be clear about what is involved which is a blessing.
When I decided to set the diary for this week I overlooked the fact that students would be coming to Reading, however, on reflection that will be no bad thing as it will allow me to look at the time impact on students of taking a day out of their weekend. Certainly compared to face to face days arranged during the week the numbers attending were lower.
The discussion forum for the diary has mainly been used to discuss the value of podcasts by students. Many point to the value of having audio versions of the printed materials either for downloading or as CDs which they can play in the car or train while travelling to work or between client sites. However, they also want the printed version too!
There is still a slow trickle of students completing the pre-course questionnaire. 553 returns have been made but this reduces to 472 after duplicates and substantially incomplete forms have been removed.
In the time I've had this week I've started to clean the pre-course questionnaire data for entry into an Access database. I've also collected the base details of all the current students from the College's student records system. Once I've learned better how to work in Access I should be able to start building a comprehensive set of records!
Tuesday, 5 February 2008
First Results of Pre-Course Questionnaire
The course and module that this research is targeting started last week on Monday 28th January. By the end of last week there were a total of 682 students enrolled of whom 63 are retaking modules. The total includes students who completed a foundation module last autumn and 195 new students who were given exemption from the prelim study. These are all first year students and the total enrolment for all three years of the course is expected to be around 1275 students.
So far over 450 students from 38 nationalities have completed the pre-course questionnaire (over 90 were spoilt) representing 67% of target students taking year one. Two thirds are male and their employment ranges from trainees up to director level.
Responses indicate that the average working week is 37.2 hours with an average of 3.9 hours paid overtime. This gives an average working day of 8.2 hours and with an average daily commuting time of one hour gives a total weekday time commitment to work of 9.2 hours.
In respect of non-work time students indicate an average of 2.4 hours per day spent in domestic duties and 2.8 hours per day participating in social activity. Outside work most students spend between one and ten hours per week online engaged in activities such as emailing, banking, shopping, networking and general reference. Potentially this adds in a further 1.5 hours per day which increases to 2 hours if the average of 30 minutes per day spent on mobile phones is added in.
Overall the averages suggest that students already have 16.4 hours committed before sleep and study are taken into account. Adding in 8 hours sleep brings the daily average to 24.4 hours leaving zero time for study!
If these averages were accepted then the conclusion must be that students are cramming all their study into weekends. Clearly this is not the case and this highlights the fruitlessness of using average values to identify the time characteristics of the ‘average’ student.
A lot more analysis still needs to undertaken but one interesting outcome is the students’ weekly use of Facebook.
Over half the students use the social networking site with a clear pattern of increase from young to old for both genders - shown in this table.
If Facebook is a valid indicator of changing times then the conclusion may be drawn that the digital natives are indeed coming - and perhaps faster than first thought!
So far over 450 students from 38 nationalities have completed the pre-course questionnaire (over 90 were spoilt) representing 67% of target students taking year one. Two thirds are male and their employment ranges from trainees up to director level.
Responses indicate that the average working week is 37.2 hours with an average of 3.9 hours paid overtime. This gives an average working day of 8.2 hours and with an average daily commuting time of one hour gives a total weekday time commitment to work of 9.2 hours.
In respect of non-work time students indicate an average of 2.4 hours per day spent in domestic duties and 2.8 hours per day participating in social activity. Outside work most students spend between one and ten hours per week online engaged in activities such as emailing, banking, shopping, networking and general reference. Potentially this adds in a further 1.5 hours per day which increases to 2 hours if the average of 30 minutes per day spent on mobile phones is added in.
Overall the averages suggest that students already have 16.4 hours committed before sleep and study are taken into account. Adding in 8 hours sleep brings the daily average to 24.4 hours leaving zero time for study!
If these averages were accepted then the conclusion must be that students are cramming all their study into weekends. Clearly this is not the case and this highlights the fruitlessness of using average values to identify the time characteristics of the ‘average’ student.
A lot more analysis still needs to undertaken but one interesting outcome is the students’ weekly use of Facebook.
Over half the students use the social networking site with a clear pattern of increase from young to old for both genders - shown in this table.
If Facebook is a valid indicator of changing times then the conclusion may be drawn that the digital natives are indeed coming - and perhaps faster than first thought!
Subscribe to:
Posts (Atom)
