The last two weeks have been slower due mainly to the mini heatwave that we experienced for the first 10 days of May. Last weekend was too hot (30 degrees) to do a lot on the computer so time was spent in the garden with a cooling hose or in the shade!!
Not that the EdD was overlooked entirely. As mentioned in my last posting, this research not only aims to understand what happens with student time during the period of study for a module but also aims to track a sample of students through a sequence of modules. Therefore the last couple of weeks have been spent looking through the initial output data to identify which students will be asked to participate in the follow up surveys.
The decision has been helped by the question asked at the end of the post-module questionnaire. Only those students who positively agreed to participate further were considered - a total of 184 students out of 310 (64%) that completed the questionnaire.
Of these, deciding which students to include has not been too difficult, and has either been based on a process of elimination or on a need to include basis. Having the detailed analysis of the IMC forums, and the Blackboard access statistics, has proved useful in determining those students who appear to have provided accurate data in their diaries about their study. In the believe that an 'honest' student will continue to offer truthful answers these students were included in the pot.
On the 'need to include' side, this stage aims to track students longitudinally over the course of their Part 1 studies. Consequently only those students who are taking the full course without retake or exemption were included. Additionally, due to the nature of the EdD those students located overseas were automatically included - not least as they are a relatively small number in comparison with UK ones. As family circumstances are a (critical) focus for the research all students indicating that they have a family aged below 16 were also included.
This provided a pool of 122 students who either invited themselves into the follow up stage or did not exclude themselves. This was considered to be too large a sample and the final step was to look through the data again to decide who to leave out. This was not done entirely scientifically but did aim to generate a representative sample by gender, age, location and study route (ie property or construction). The aim was to end up with a sample of 50 students. In the end 62 students have been identified which represents about 20% of the main group of 295 students. The larger number allows for casualties along the way.
The tracking sample will be emailed later this week and invited to complete a follow up 'tracking' questionnaire. This aims to collect changes in time use and life circumstances from the students and will be repeated after the next two modules as well. It contains several questions identical to those in the post IMC questionnaire allowing comparison to be made. It also includes questions that drill deeper than the previous one in areas related to time and the course, study, personal circumstances and employment issues. Lastly it feeds back the summary weekly time commitment from the student's own diary and asks them to update on how their use of time has changed.
I'm not sure that I want, or even need, more data, however, my objectives will not be met unless I can see how students use of time changes as they come up against different life events.
Onward and upward!!
Sunday, 18 May 2008
Monday, 5 May 2008
SPSS and all that!
Since my last post I have looked again at the diary data and find that there are a further 74 sets of data which are complete except for the post-module questionnaire. Therefore I have added them to the collection making 295 sets of partial or complete data. Each set has been printed out on 6 sides of A4 consuming reams of paper and several printer cartidges. The whole collection now fills six 2 inch (5 cm) ring binders!
Apart from printing and filing, my last couple of weekends have been taken up with re-acquainting myself with SPSS. I haven't really used it since the taught module on methods of enquiry (about 2004) so am somewhat rusty- also of course it is now in a new version. Thank goodness for up-to-date textbooks! SPSS 15 Made Simple by Paul Kinnear and Colin Gray is particularly helpful.
Having spent many hours checking through the Excel data relating to students and the Pre-Course questionnaire to ensure that the variables are complete and the data in the right form to import I have hit a problem today with licencing. Basically the student version that I bought tells me that the licence has expired which, as it is not yet the end of the academic year, it should not.
So rather than go further I have passed the query back to the help desk and will spend the afternoon in the garden. I'm not unhappy about this as the sun has finally come out, but it is a bit of an irritant as I had planned on getting the data into SPSS by the end of this holiday weekend.
On another front, in the next week or so I have to decide on the group of students that I will track over the coming months. Not only do I need to identify the significant profiles but I also need to prepare the questionaire for their feedback.
Hey Ho - never a dull moment in research!!
Apart from printing and filing, my last couple of weekends have been taken up with re-acquainting myself with SPSS. I haven't really used it since the taught module on methods of enquiry (about 2004) so am somewhat rusty- also of course it is now in a new version. Thank goodness for up-to-date textbooks! SPSS 15 Made Simple by Paul Kinnear and Colin Gray is particularly helpful.
Having spent many hours checking through the Excel data relating to students and the Pre-Course questionnaire to ensure that the variables are complete and the data in the right form to import I have hit a problem today with licencing. Basically the student version that I bought tells me that the licence has expired which, as it is not yet the end of the academic year, it should not.
So rather than go further I have passed the query back to the help desk and will spend the afternoon in the garden. I'm not unhappy about this as the sun has finally come out, but it is a bit of an irritant as I had planned on getting the data into SPSS by the end of this holiday weekend.
On another front, in the next week or so I have to decide on the group of students that I will track over the coming months. Not only do I need to identify the significant profiles but I also need to prepare the questionaire for their feedback.
Hey Ho - never a dull moment in research!!
Monday, 21 April 2008
Initial Data Collection Complete!
Yesterday I drew a line under the initial phase of the data collection when I closed the online questionnaires to further responses.
To summarise the target students for this research:
705 students are registered to study the year 1 course in 2008, and thus were invited to complete the Pre-Course questionnaire. 567 responses were returned to the questionnaire which yielded 506 usable sets of data once duplicates were removed. As a principle the latest version of the questionnaire has been included unless this was a blank response, in which case the next earliest that included answers was taken. Overall this equates to a 72% response rate to the Pre-Course Questionnaire.
470 students are registered to take the Information Management and Control module which the initial phase has focused on. Of these 74 students were retaking the module after failing or deferring it in a previous year. 381 students submitted a copy of the diary that they kept for week 2 of the module, of which 363 were usable for research purposes. This equates to a response rate of 77% for the diary.
In respect of the module assessment 447 students submitted their first assignment and 438 submitted assignment 2, a submission rate of 95% and 93% respectively. In terms of module outcome this results in 430 students passing the module - a pass rate of 91%.
348 students submitted a response to the Post-Module questionnaire. After duplicates were removed (following the same criteria as for the Pre-Course questionnaire) this yielded 310 usable responses - a response rate of 66%.
When these various sets of data are combined, the result is a total of 221 complete sets of data that include responses to the two questionnaires, submission of the diary as well as details of student's assessment marks and their participation in the VLE forums. This equates to 47% of the module population, however, as only 2 students retaking the module submitted a complete set of data this is effectively 55% of new students taking the module.
In respect of profile of the full sets collected, 56% are from male students (compared with 65% across all students taking the module), which would appear to be representative. What appears less representative is that 99% of the students submitting complete sets of data are passing the module whereas only 1% (2 students) have failed it. This does suggest that a proportion of students (up to 45) may already have dropped out or deferred before the module ended. What is not clear is the extent that time may have caused, or contributed, to this.
The next phase is to analyse the data. All 221 sets of data are now printed and a visual examination will be the first step. This should reveal those factors which seem to contribute to time pressures, and so will be extracted into SPSS for more detailed analysis.
Never a dull day!!
To summarise the target students for this research:
705 students are registered to study the year 1 course in 2008, and thus were invited to complete the Pre-Course questionnaire. 567 responses were returned to the questionnaire which yielded 506 usable sets of data once duplicates were removed. As a principle the latest version of the questionnaire has been included unless this was a blank response, in which case the next earliest that included answers was taken. Overall this equates to a 72% response rate to the Pre-Course Questionnaire.
470 students are registered to take the Information Management and Control module which the initial phase has focused on. Of these 74 students were retaking the module after failing or deferring it in a previous year. 381 students submitted a copy of the diary that they kept for week 2 of the module, of which 363 were usable for research purposes. This equates to a response rate of 77% for the diary.
In respect of the module assessment 447 students submitted their first assignment and 438 submitted assignment 2, a submission rate of 95% and 93% respectively. In terms of module outcome this results in 430 students passing the module - a pass rate of 91%.
348 students submitted a response to the Post-Module questionnaire. After duplicates were removed (following the same criteria as for the Pre-Course questionnaire) this yielded 310 usable responses - a response rate of 66%.
When these various sets of data are combined, the result is a total of 221 complete sets of data that include responses to the two questionnaires, submission of the diary as well as details of student's assessment marks and their participation in the VLE forums. This equates to 47% of the module population, however, as only 2 students retaking the module submitted a complete set of data this is effectively 55% of new students taking the module.
In respect of profile of the full sets collected, 56% are from male students (compared with 65% across all students taking the module), which would appear to be representative. What appears less representative is that 99% of the students submitting complete sets of data are passing the module whereas only 1% (2 students) have failed it. This does suggest that a proportion of students (up to 45) may already have dropped out or deferred before the module ended. What is not clear is the extent that time may have caused, or contributed, to this.
The next phase is to analyse the data. All 221 sets of data are now printed and a visual examination will be the first step. This should reveal those factors which seem to contribute to time pressures, and so will be extracted into SPSS for more detailed analysis.
Never a dull day!!
Sunday, 6 April 2008
Data Collected?
Maybe this scenario is familiar to you. You have a period of free time coming up and you plan to get a lot of work / research / writing done - then life intervenes!! This was the story of my Easter break. I had some leave to use up before the end of March so booked to take the week off after Easter. My plan was to sort out the data and get it into a form ready for SPSS. All went well until Easter Sunday and then the dreaded 'flu virus struck and that was me finished for a whole week! It is a long time since I've been so badly affected to the point that I have not even wanted to switch the computer on!!
So now I am trying to catch up with my schedule. In reality I am still collecting data from the students. Despite reminders there are still a number who have not sent me their diary, however, I reckon this reflects the accuracy (or lack of it) so I am no longer pushing for these. Actually I have done pretty well and have 358 diaries to work with.
I am still pressing students to complete the post-module questionnaire as the response has not been as good as I had hoped for. 327 have so far completed it which is not bad but leaves quite a few incomplete sets of data (pre-course questionnaire, diary, post-course questionnaire) which if I can get a further 80 or so to submit will be good.
I have now finished analysing the module forums which give me actual details of messages posted and replied to, and at what time on which day these were made. Combined with the Blackboard stats, this not only allows me to compare the actual access and activity against the diaries but will also give me a picture of whether a student was posting to the study schedule and, if not, how far behind they may have got.
The last set of data that I have yet to complete is the assignment mark data. This is now waiting for one or two slow tutors to return their assignment 2 results. Once in I should be in a position to start printing out sets of results per student. As of today I have around 190 sets of complete data but with luck (and the cooperation of the students) I am hoping to get this up to around 240 students ie half the students taking the module.
The last three months have been very pressurised in terms of data collection. I am now looking forward to a period of analysis and drawing interim conclusions from the data.
So now I am trying to catch up with my schedule. In reality I am still collecting data from the students. Despite reminders there are still a number who have not sent me their diary, however, I reckon this reflects the accuracy (or lack of it) so I am no longer pushing for these. Actually I have done pretty well and have 358 diaries to work with.
I am still pressing students to complete the post-module questionnaire as the response has not been as good as I had hoped for. 327 have so far completed it which is not bad but leaves quite a few incomplete sets of data (pre-course questionnaire, diary, post-course questionnaire) which if I can get a further 80 or so to submit will be good.
I have now finished analysing the module forums which give me actual details of messages posted and replied to, and at what time on which day these were made. Combined with the Blackboard stats, this not only allows me to compare the actual access and activity against the diaries but will also give me a picture of whether a student was posting to the study schedule and, if not, how far behind they may have got.
The last set of data that I have yet to complete is the assignment mark data. This is now waiting for one or two slow tutors to return their assignment 2 results. Once in I should be in a position to start printing out sets of results per student. As of today I have around 190 sets of complete data but with luck (and the cooperation of the students) I am hoping to get this up to around 240 students ie half the students taking the module.
The last three months have been very pressurised in terms of data collection. I am now looking forward to a period of analysis and drawing interim conclusions from the data.
Saturday, 15 March 2008
.... and Even More Data
As predicted this week has seen a deluge of diaries hit my inbox. Overnight from Sunday to Monday 75 arrived and this morning I have 326 diaries in total, hopefully with more to come.
I have now got opening the diary file, removing the password protection, copying and pasting the data into the master file and then saving the original diary down to a fine art. I reckon each now takes me about 2 minutes - so getting on for 3 hours work just to collect the data.
The post-module questionnaire went live last Sunday and already 200 students have completed it. All being well I am anticipating having a complete set of data (questionnaires and diary) for around 300 students. More would be nice!
One of the benefits of being in the UK is the ability to pop into IOE. I was in London for a workshop on advanced scholarship 10 days ago and arranged to have 20 minutes with my supervisor on the way home. It really is helpful to use occassions like that to take stock and to articulate where you are at with someone. Makes me think that a 'buddy' system could work equally well for times in between.
Incidentally it was graduation day at IOE so nice to see the students dressed up in all their finery and caps and gowns. Have to say the EdD robes look well worth the time and effort:-)
I have now got opening the diary file, removing the password protection, copying and pasting the data into the master file and then saving the original diary down to a fine art. I reckon each now takes me about 2 minutes - so getting on for 3 hours work just to collect the data.
The post-module questionnaire went live last Sunday and already 200 students have completed it. All being well I am anticipating having a complete set of data (questionnaires and diary) for around 300 students. More would be nice!
One of the benefits of being in the UK is the ability to pop into IOE. I was in London for a workshop on advanced scholarship 10 days ago and arranged to have 20 minutes with my supervisor on the way home. It really is helpful to use occassions like that to take stock and to articulate where you are at with someone. Makes me think that a 'buddy' system could work equally well for times in between.
Incidentally it was graduation day at IOE so nice to see the students dressed up in all their finery and caps and gowns. Have to say the EdD robes look well worth the time and effort:-)
Saturday, 1 March 2008
Data, Data and More Data!
Funny how time disappears so quickly!! Over the last couple of weeks I've been processing the diaries as they come in. I now have over 160 sets of data but the real rush will come in a weeks time when students submit the assignment that the diary is attached to. My guess is that most are waiting until after they have finished writing the assignment before they send the diary to me. 479 students are taking the Information Management module so I've probably only received around a third of the diaries so far!
One tip to pass on from the experience with the diaries is to write down safely any passwords used. I password protected the diary spreadsheets leaving just the cells requiring entry to be accessible. The spreadsheet collected all the export info into a single hidden column of 1133 items so that the cut and paste into the analysis spreadsheet would be straightforward. Only when I went to unprotect the first diary did I discover that the password I thought I'd used did not work and remissly I had not written it down anywhere. You can imagine the frustration trying different words to access the data without success. Fortunately a search on the web revealed a handy bit of software that can be loaded and will unprotect spreadsheets. I don't know how it works but it does and now I have my data - relief :-).
As indicated I am collecting the diary results into an analysis spreadsheet which allows me to analyse each line of data to get summaries and averages. It also allows me to build a proportional chart from all the responses so that I can compare an individual student with the 'average'. From this comparison I am hoping to identify like groups of students who have similar time profiles, and who can then be followed up in greater detail.
The other thing I've been doing since the last post is to analyse the students' contributions to the online discussion forums. A couple of years ago I wrote a spreadsheet that will produce a simple analysis of messages and tells me who has posted with date and time, whether the post was an original message or a reply to another participant and how many times the message was read. From this I will be able to verify the online (VLE) entries in students' diaries and gives a more accurate profile than the statistics of hits that Blackboard provides. It is a bit lengthy with 24 tutor groups to open and then to collect, copy and paste the messages from each of the 10 forums but it does the job.
Gradually all the different sources of data are being combined into summary pages for each student. Once this is complete I will have a better idea of the full set of variables available for detailed analysis. In the meantime my next job is to prepare the post-module questionnaire ready for the 10th March when students submit the end assignment. No rest for the wicked as they say!!
One tip to pass on from the experience with the diaries is to write down safely any passwords used. I password protected the diary spreadsheets leaving just the cells requiring entry to be accessible. The spreadsheet collected all the export info into a single hidden column of 1133 items so that the cut and paste into the analysis spreadsheet would be straightforward. Only when I went to unprotect the first diary did I discover that the password I thought I'd used did not work and remissly I had not written it down anywhere. You can imagine the frustration trying different words to access the data without success. Fortunately a search on the web revealed a handy bit of software that can be loaded and will unprotect spreadsheets. I don't know how it works but it does and now I have my data - relief :-).
As indicated I am collecting the diary results into an analysis spreadsheet which allows me to analyse each line of data to get summaries and averages. It also allows me to build a proportional chart from all the responses so that I can compare an individual student with the 'average'. From this comparison I am hoping to identify like groups of students who have similar time profiles, and who can then be followed up in greater detail.
The other thing I've been doing since the last post is to analyse the students' contributions to the online discussion forums. A couple of years ago I wrote a spreadsheet that will produce a simple analysis of messages and tells me who has posted with date and time, whether the post was an original message or a reply to another participant and how many times the message was read. From this I will be able to verify the online (VLE) entries in students' diaries and gives a more accurate profile than the statistics of hits that Blackboard provides. It is a bit lengthy with 24 tutor groups to open and then to collect, copy and paste the messages from each of the 10 forums but it does the job.
Gradually all the different sources of data are being combined into summary pages for each student. Once this is complete I will have a better idea of the full set of variables available for detailed analysis. In the meantime my next job is to prepare the post-module questionnaire ready for the 10th March when students submit the end assignment. No rest for the wicked as they say!!
Sunday, 17 February 2008
Getting Down to Stats
This last week has been one of facing up to the volume of data that this research is generating.
Completion of the pre-course questionnaire continues at a trickle with 561 students having now completed it yielding 481 usable sets of data. So far 70 students have returned their completed diaries. Interestingly some appear to have chosen to give someone else's ID for the diary!
How to analyse the diary content has been high in my thoughts. During the teaching week in London last week I had a one-to-one session with one of the lecturers who knows about stats and SPSS. It was really helpful to 'voice' the research and data issues, and the result is I am much clearer about the form in which the data needs to be recorded .
One issue that stats cannot resolve is that of sequencing ie the order and times that students spend on different activities during a day. My thinking at the moment is that this will have to be looked at visually. I have therefore created a spreadsheet that shows the time students spend on the four main activities (resting, working, studying and other) as colour coded 10 minute blocks. Already this serves to show whether an individual is blocking longer periods of time or grabbing short sessions, and whether this is happening during the week or at weekends.
Another tool I have created is a proportional 2-D chart that summarises the student's week, and is based on one illustrated by Gershuny (see previous blog). Created in Excel, it shows the proportion of time spent on the 4 main activities as a grid with columns of width equal to the percentage of time spent on that group of activities. Each column then sub-divides into the time activities within that group with the height of each block proportional to the time spent on it. One idea for using this is to print each chart on acetate and to overlay them. This should help to identify common patterns of time usage (and may be simpler than trying to do multi-variate analysis!!). I'll let you know how I get on.
I'm one of those people who is more than happy crunching number but still get lost with the more advanced stats - mainly due to not using the techniques often enough. For anyone in a similar position, or even for those who are unsure about statistics, can I recommend a book by Reva Berman Brown and Mark Saunders called Dealing with Statistics. The great thing is that there are no formulae or numbers included at all!! What they do is to talk you through the basics of statistics and what techniques can / should be used for different purposes. It is only a short book so doesn't take long to read but quickly gives the sorts of reminders that a rusty mind like mine needs.
Completion of the pre-course questionnaire continues at a trickle with 561 students having now completed it yielding 481 usable sets of data. So far 70 students have returned their completed diaries. Interestingly some appear to have chosen to give someone else's ID for the diary!
How to analyse the diary content has been high in my thoughts. During the teaching week in London last week I had a one-to-one session with one of the lecturers who knows about stats and SPSS. It was really helpful to 'voice' the research and data issues, and the result is I am much clearer about the form in which the data needs to be recorded .
One issue that stats cannot resolve is that of sequencing ie the order and times that students spend on different activities during a day. My thinking at the moment is that this will have to be looked at visually. I have therefore created a spreadsheet that shows the time students spend on the four main activities (resting, working, studying and other) as colour coded 10 minute blocks. Already this serves to show whether an individual is blocking longer periods of time or grabbing short sessions, and whether this is happening during the week or at weekends.
Another tool I have created is a proportional 2-D chart that summarises the student's week, and is based on one illustrated by Gershuny (see previous blog). Created in Excel, it shows the proportion of time spent on the 4 main activities as a grid with columns of width equal to the percentage of time spent on that group of activities. Each column then sub-divides into the time activities within that group with the height of each block proportional to the time spent on it. One idea for using this is to print each chart on acetate and to overlay them. This should help to identify common patterns of time usage (and may be simpler than trying to do multi-variate analysis!!). I'll let you know how I get on.
I'm one of those people who is more than happy crunching number but still get lost with the more advanced stats - mainly due to not using the techniques often enough. For anyone in a similar position, or even for those who are unsure about statistics, can I recommend a book by Reva Berman Brown and Mark Saunders called Dealing with Statistics. The great thing is that there are no formulae or numbers included at all!! What they do is to talk you through the basics of statistics and what techniques can / should be used for different purposes. It is only a short book so doesn't take long to read but quickly gives the sorts of reminders that a rusty mind like mine needs.
Subscribe to:
Posts (Atom)
