Rants, Raves, and Rhetoric v4

Weblogic Diagnostics

I noticed one the nodes in a development cluster was down. So I started it again. The second start failed, so I ended up looking at logs to figure out why. The error in the WebCTServer.000000000.log said:

weblogic.diagnostics.lifecycle.DiagnosticComponentLifecycleException: weblogic.store.PersistentStoreException: java.io.IOException: [Store:280036]Missing the file store file “WLS_DIAGNOSTICS000001.DAT” in the directory “$VISTAHOME/./servers/$NODENAME/data/store/diagnostics”

So I looked to see if the file was there. It wasn’t.

I tried touching a file at the right location and starting it. Another failed start with a new error:

There was an error while reading from the log file.

So I tried copying to WLS_DIAGNOSTICS000002.DAT to WLS_DIAGNOSTICS000001.DAT and starting again. This got me a successful startup. Examination of the WLS files revealed the the 0 and 1 files have updated time stamps while the 2 file hasn’t changed since the first occurance of the error.

That suggests to me Weblogic is unaware of the 2 file and only aware of the 0 and 1 files. Weird.

At least I tricked the software into running again.

Some interesting discussion about these files.

  1. Apparently I could have just renamed the files. CONFIRMED
  2. The files capture JDBC diagnostic data. Maybe I need to look at the JDBC pool settings. DONE (See comment below)
  3. Apparently these files grow and add a new file when it reaches 2GB. Sounds to me like we should purge these files like we do logs. CONFIRMED
  4. There was a bug in a similar version causing these to be on by default.

Guess that gives me some work for tomorrow.
🙁

Ezra S F

, , , , , , , , , , , ,

One response to “Weblogic Diagnostics”

  1. Ez Avatar
    Ez

    Found a node with 4 of these files. Stopped the node, renamed the files to have .old at the end, and started up the node. It started fine so I’ve deleted the files. #1 and #3 confirmed.

    #2 sorta confirmedOnly thing checked in the JDBC Diagnostics is “Profile Connection Leak”:

    Connection Leak (PROFILE_TYPE_CONN_LEAK_STR)

    Enable connection leak profiling to collect information about threads that have reserved a connection from the data source and the connection leaked (was not properly returned to the pool of connections). This profile information can help determine which applications are not properly closing JDBC connections. The record contains the following information:

    • PoolName – name of the data source to which this connection belongs
    • ID – connection ID
    • User – stack trace of the thread waiting for the connection
    • Timestamp – time stamp showing when the connection leak was detected

    Source: Monitoring WebLogic JDBC Resources

    Reply

Leave a Reply