Why can I only see test answers if test is completed?

struitt
struitt Community Member Posts: 137 ♪ Opening Act ♪
Hi all.

I have a test, made in Lectora 18 and using Saba as the LMS and taken in Chrome. If the learner completes the test then I can see all of their correct/incorrect answers by running a report in Saba.

However, if the learner exits the course early, let's say, 30% into it, the report in Saba simply lists it as 'incomplete' with zero answers/responses. How I can I get it to report that 30%? We want to see what questions were attempted, even if the learner doesn't finish the test.

Answers

  • wheels
    wheels Florida USACommunity Member, Administrator, Moderator, Rockstar Manager Posts: 595 eLearning ROCKSTAR Admin Team
    The to report those questions the Process Test action must be run. You might be able to do something in your course to have the learner submit the test.

    One way might be to have a Cancel Test button that really submits the test.

    1. Add a Cancel button
    2. Add an action to the Cancel button to modify a variable called TestCancelled to true
    3. Add an action to Process the Test
    4. On the results page have a Go to action that checks if TestCancelled is true to send your learner to an appropriate place
    5. On the first test page remember to set TestCancelled to false (in case the learner tries again)

  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    Thank you wheels.

    That is close to what we eventually came up with, but have yet to implement. I will add some of your input as well.

    Thanks again!
  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    Plot twist...the course is randomized. The only [navigational] actions allowed in a randomized test are back/next, according to when I do an 'error check.' So I can't publish it with a Cancel button that does anything other than take the learner to the back/next page.
  • wheels
    wheels Florida USACommunity Member, Administrator, Moderator, Rockstar Manager Posts: 595 eLearning ROCKSTAR Admin Team
    @struitt I was thinking about that but wasn't sure that was your case. To process the test in a random test you would create a "cancelled" page outside of the test and have the cancel button navigate to that page. Navigating outside of the test will cause the test to be processed. I'm not sure if the test processing will override the navigation to the cancelled page, but you can adjust for that. Hope this helps, please post what you find.
  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    edited May 24
    I tried that. Anything other than a 'next/back' action in a randomized test, such as a 'go to cancel page that is outside of the test,' triggers the error and will not allow me to SCORM. I'm pretty sure the main, and possibly only, issue with all of this working is the 'randomization' part. I tried it with the test being randomized and it seemed to work fine.

    We have abandoned that idea and will be trying something with less detail and moving parts.
  • andrew-robertson
    andrew-robertson Community Member Posts: 196 ☆ Superstar ☆
    If i can provide an idea.
    1. Set TEST behaviour for On Completed/Passed to Next Page and On Cancelled/Failed to Next Page.
    2. Create action group to include 3 actions: (1) Hide submit button, (2) Process question, (3) Process Test/Survey (delay: 1.0s).
    3. btn_Submit runs the action group with the above mentioned actions ONLY if the question is not empty.
    4. Create your End of Exam page on the outside of the randomized TEST SECTION which would have your exam score.
    See the image below for my current setup, but it doesn't have the submission after every question. If you'd like me to make a sample 5Q exam, I can do that for you.

    Andrew R.
    Developing content since 2010 using a mixed bag of tools.
  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    I like the way you think Andrew. Something like that did cross my mind, but we are under a time crunch so we went another route. However, I think I will make a copy of my course and run with your idea, even if it will only be used in future updates. Thank you.
  • wheels
    wheels Florida USACommunity Member, Administrator, Moderator, Rockstar Manager Posts: 595 eLearning ROCKSTAR Admin Team
    Ok I will be interested to see how you solve it. In the random test you might be able to use a JavaScript action to call the method to process the test.

    ProcessTest(1)

    I will ask Product Management to write up an enhancement issue to send question interaction data when the question is submitted. @chris_willis
  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    edited May 28
    I think...pending LMS testing with the reports...I think I got it.
    • I created a new page (named 'roundabout') at the end of the course (outside of the test).
    • I put a 'Leave Early' button at the test level with an action to process the test. This way the learner could leave on any page within the test.
    • I made the test pass/fail actions both go to the 'roundabout' page.
    • Four actions on the 'roundabout' page determined where the learner would go: pass page (based on score); fail page (based on score); out of time page (based on variable tripped when timer expired); or left early page (determined by variables set earlier)
    In early tests I could see exactly how many questions I answered, regardless of where/how I exited the course, along with the answer I selected. That's what we are going for. The final test is going to be what our LMS can provide in terms of reporting.

    Big thanks to those of you that responded and offered advice.
  • andrew-robertson
    andrew-robertson Community Member Posts: 196 ☆ Superstar ☆
    edited May 28
    Funny enough, we did something similar to support the end of exam answer submission but it wasn't to support the need to leave early. We actually don't support leaving an exam early, it's all or nothing in our business processes because they are certification exams.

    My only concern with your process is the what if the user doesn't click the leave early button but clicks the X button in the upper right corner of the web browser? Sadly this is really a human factor that causes us a lot of grief for me because that's what we all do as humans when not in class and just browsing the web.

    But never the less, I do like the thinking process - many ways to program and make things work!!
    Andrew R.
    Developing content since 2010 using a mixed bag of tools.
  • struitt
    struitt Community Member Posts: 137 ♪ Opening Act ♪
    Exiting using the browser's "X" was discussed and is a concern for us as well. At this point we are just saying it is the nature of the beast and will deal with it on a case-by-case basis.

    We also made the 'Leave Early' button stand out so hopefully, if they do choose to leave early, they will use the bright button specifically labeled 'Leave Early.' In addition to that, at the beginning of the course, there are instructions to not use the browser's "X," but we know some will anyway.
  • andrew-robertson
    andrew-robertson Community Member Posts: 196 ☆ Superstar ☆
    I love answering management that it is the nature of the beast... such a great answer and it gets me out of a bunch of sticky situations.
    Andrew R.
    Developing content since 2010 using a mixed bag of tools.
  • chris_willis
    chris_willis Product Director, Lectora, eLearning Brothers West MichiganRockstar Manager Posts: 69 eLearning ROCKSTAR Stage Manager
    I love the idea-sharing in this thread! I'll add it also helps to make sure your course nav is hard to miss, including a big X button to exit.

    From an ID perspective, I really HATE to force a nav tutorial at the start of every course, but it is standard practice to include a caution or note any time there is a risk of data loss:

    IMPORTANT: Do not use your browser back, forward, refresh, or exit while taking this course. Always use the course navigation and exit buttons to ensure your progress and score are recorded.

    Depending on the material and workplace culture, I'd be tempted to add something sassy about what a shame it would be if we had to force them to take the same course twice. With some material, that threat could make learners sit up and take notice! :wink:

    ----------------------------------------------
    Chris Willis, Product Director, Lectora
    ◖⚆ᴥ⚆◗ "Where there's a Willis, There's a Way!"
  • zenabartell
    zenabartell Community Member, Lectora® Accessibility Group (LAUG) Member, eLearning Brothers® Partner Posts: 1 ☆ Roadie ☆
    Thank you very much for this useful article. I like it.. Keep it up and never miss a chance to grow. slope unblocked