Yesterday a Blackboard Vista node went unresponsive but recovered. Then a second and third did the same. This was odd as only the test which logs into the node reported a failure. Normally, in severe unresponsive cases the Weblogic monitor either reports either one of the parameters we watch is very high or fails because it was unable to connect to the Weblogic.

The graphs for CPU showed 4 periods of sustained activity near 100% for longer than 10 minutes. In each case, when the CPU dropped there were multiple instances of editAssignmentSubmission.dowebct (not in the PowerSight tracked actions?) just completed. Normally when these are browser resubmissions the logged times are near identical but the time-to-process is about 300 seconds apart. In these cases the logged times and time-to-process are within seconds of each other.

I suspect this means the users (in this case students) are hitting the Submit button repeatedly. Maybe resources devoted to handling process0 are diverted to handling processes 1-5 and causing it to take longer not shorter.

Another possibility is the instructor is having the students upload something massive which takes forever to load into the database. Movies and music take up a ton of space. Maybe I could look at the graphs on storage space and see how quickly the space was consumed compared to control time frames (similar amount of activity).