Charles Lee - 18/Mar/08 03:45 PM Scott's looking into it

Ryan Morgan - 19/Mar/08 09:38 AM It appears the JSON output is correct based on the contents of the DownMetricsCache. In places we see this the logs are full of: 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 405264 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 404532 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 404388 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 405318 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 403686 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 403272 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 403866 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 405084 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 405300 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 403704 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 404442 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 404082 in down metrics cache. 2008-03-19 00:20:46,456 WARN [DownMetricsCalculator1] [org.hyperic.hq.measurement.server.session.DataManagerEJBImpl] No availability metric for for id 401754 in down metrics cache.

Charles Lee - 19/Mar/08 12:29 PM Ultimately the problem is the down metrics cache size. Due to the code path, we are pushing every single availability metric into the cache, meaning that the size needs to be greater than the number of resources. However, code has been changed to reduce unnecessary insertion, so the cache can theoretically be slightly larger than the maximum number of down resources at any time. However, it's still possible in a system that has a lot of churn for things to be pushed out of the cache, so it's still recommended that we scale the cache size with the environment.

Kashyap Parikh - 24/Mar/08 03:39 PM Looks good. Performed platform/server/services up/down tests and resources are showing up fine on currently down page after page refresh. Zevent queue also looks fine after server running for 72 hours. ZEvent Listener Diagnostics:     EventClass: class org.hyperic.hq.measurement.server.session.DownMetricZevent         DownMetricsCalculator max=0.00 avg=0.00 num=108 Zevent Registered Buffers:     DownMetricsCalculator size=0                                        max=1820.00 avg=31.56 num=108