Support Migration Notice: To update migrated JIRA cases click here to open a new case use www.vmware.com/go/sr | vFabric Hyperic 5.7.0 is Now Available

Hyperic HQ

Measurement Scheduling still occurs on VMs when ESX host moved from one vCenter server to another vCenter server

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Deferred
  • Affects Version/s: 4.4
  • Fix Version/s: None
  • Component/s: Plugins
  • Environment:
    HQ-server 4.4.0-EE-1457 running on CentOS 5.4
    HQ-agent 4.4.0-EE-1457 running on Windows 2008
     - VMware vCenter Server 4.0 running with HQ-agent
  • Case Links:
    none
  • Regression:
    No
  • Tags:

Description

Measurement Scheduling still occurs on VMs when ESX host moved from one vCenter server to another vCenter server.

When an ESX host has been moved to one vCenter server (original) to another vCenter server (new), the measurement scheduling should be removed from the orignal vCenter server.

For a previously suspended VM, the agent.log on the original vCenter server reports:
010-06-04 16:47:00,169 ERROR [ScheduleThread] [ScheduleThread] Measurement plugin error while processing Metric 'VMware vSphere VM:vm:url=https%3A//10.16.16.161/sdk,user=mmennis,pass=*******,vm=vmwin7-hqqa1:Availability'
org.hyperic.hq.product.PluginException: VirtualMachine/vmwin7-hqqa1: not found
at org.hyperic.hq.plugin.vsphere.VSphereHostCollector.init(VSphereHostCollector.java:114)
at org.hyperic.hq.product.Collector.getValue(Collector.java:512)
at org.hyperic.hq.product.MeasurementPlugin.getValue(MeasurementPlugin.java:445)
at org.hyperic.hq.product.MeasurementPluginManager.getPluginValue(MeasurementPluginManager.java:176)
at org.hyperic.hq.product.MeasurementPluginManager.getValue(MeasurementPluginManager.java:274)
at org.hyperic.hq.measurement.agent.server.ScheduleThread.getValue(ScheduleThread.java:298)
at org.hyperic.hq.measurement.agent.server.ScheduleThread.collect(ScheduleThread.java:387)
at org.hyperic.hq.measurement.agent.server.ScheduleThread.collect(ScheduleThread.java:344)
at org.hyperic.hq.measurement.agent.server.ScheduleThread.collect(ScheduleThread.java:490)
at org.hyperic.hq.measurement.agent.server.ScheduleThread.run(ScheduleThread.java:512)
at java.lang.Thread.run(Unknown Source)
Caused by: org.hyperic.hq.plugin.vsphere.ManagedEntityNotFoundException: VirtualMachine/vmwin7-hqqa1: not found
at org.hyperic.hq.plugin.vsphere.VSphereUtil.find(VSphereUtil.java:134)
at org.hyperic.hq.plugin.vsphere.VSphereCollector.getManagedEntity(VSphereCollector.java:91)
at org.hyperic.hq.plugin.vsphere.VSphereHostCollector.init(VSphereHostCollector.java:112)
... 10 more

For VMs that were powered on during the move, the agent.log on the original vCenter server reports:
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:UTILIZATION] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:cpu.usage.average' -> 2 timestamp=1275695400513
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:UTILIZATION] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:disk.usage.average' -> 25 timestamp=1275695400513
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:UTILIZATION] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:mem.vmmemctl.average' -> 0 timestamp=1275695400513
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:UTILIZATION] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:mem.usage.average' -> 11 timestamp=1275695400513
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:UTILIZATION] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:net.usage.average' -> 0 timestamp=1275695400513
2010-06-04 16:50:00,513 DEBUG [ScheduleThread] [ScheduleThread] [1:12866:AVAILABILITY] Metric='VMware vSphere VM:vm:url=https%3A//10.150.29.72/sdk,user=administrator,pass=*******,vm=vmcent54x64-db297:Availability' -> 1 timestamp=1275695400513

Expected Result:
When an ESX host is moved from the original server, all collections and measurement scheduling for the associated hosted VMs should be removed.

Actual Result:
Measurement scheduling is still being reported on VMs belonging to moved ESX host.

Steps to Reproduce:
1. Log into vSphere Client connecting to new vCenter server
2. Add host to vCenter server which is already under control on original vCenter server
3. Restart hq-agent on new vCenter server
4. Log into HQ Server and go to HQ vSphere
5. Note the 'moved' ESX host is now recorded under new vCenter server
6. Log into original vCenter server
7. Restart hq-agent on original vCenter server
8. Examine agent.log file
9. See information posted about VMs that are hosted on ESX host which has been moved to new vCenter server

Work around:
None

Issue Links

Activity

Hide
Patrick Nguyen added a comment -

depends on HHQ-4058

Show
Patrick Nguyen added a comment - depends on HHQ-4058
Hide
Frederic Calindas added a comment -

The Error is no longer being reported; however, the Measurement Schedule is still occuring after a Host is moved to another vCenter server.

The ESX host, in this case, was moved well before the current time displayed in the logs:

The agent.log shows:
2010-07-02 17:07:28,847 DEBUG [CollectorThread] [Collector] {hostname=10.16.16.55, uuid=34313236-3435-5553-4538-30374e39414a, user=administrator, url=https://10.150.29.72/sdk, pass=*******} ran 2 seconds ago, consumed 27 seconds ago, 1min itv: collecting.

The server.log reports:
2010-07-02 17:07:34,308 WARN  [Thread-3469] [org.hyperic.hq.measurement.server.session.ReportProcessorEJBImpl@229] measurement (id=84901, name=Availability) was sent to the HQ server from agent (agentToken=1277485444414-4915040392276970568-3290688068780227864, name=10.150.29.72, port=2175) but resource (id=18691, name=10.16.16.55 {34313236-3435-5553-4538-30374e39414a}) is not associated  with that agent.  Dropping measurement.

So it appears the agent is still trying to get measurements and sending them to the server but the server is rejecting them.

Show
Frederic Calindas added a comment - The Error is no longer being reported; however, the Measurement Schedule is still occuring after a Host is moved to another vCenter server. The ESX host, in this case, was moved well before the current time displayed in the logs: The agent.log shows: 2010-07-02 17:07:28,847 DEBUG [CollectorThread] [Collector] {hostname=10.16.16.55, uuid=34313236-3435-5553-4538-30374e39414a, user=administrator, url=https://10.150.29.72/sdk, pass=*******} ran 2 seconds ago, consumed 27 seconds ago, 1min itv: collecting. The server.log reports: 2010-07-02 17:07:34,308 WARN  [Thread-3469] [org.hyperic.hq.measurement.server.session.ReportProcessorEJBImpl@229] measurement (id=84901, name=Availability) was sent to the HQ server from agent (agentToken=1277485444414-4915040392276970568-3290688068780227864, name=10.150.29.72, port=2175) but resource (id=18691, name=10.16.16.55 {34313236-3435-5553-4538-30374e39414a}) is not associated  with that agent.  Dropping measurement. So it appears the agent is still trying to get measurements and sending them to the server but the server is rejecting them.
Hide
Kashyap Parikh added a comment -

Frederic, can you retest this use case.

Show
Kashyap Parikh added a comment - Frederic, can you retest this use case.
Hide
Frederic Calindas added a comment -

Verified in 4.5.0-EE-143.

When a ESX host is mvoed from one vCenter server to another, measurements are still trying to be collected as the Host/VMs are still present in HQ vSphere though they show in the inventory tree as gray and italics. In the vSphere client (connected to the vCenter Server), they show as disconnected.

The agent.log of the vCenter server where the ESX host was moved from shows one of the VMs still collecting measurements:

2010-10-25 17:52:22,226 DEBUG [CollectorThread] [Collector] {vm=FC-TemplateTest-VM1, uuid=4206ed55-040d-1c00-780d-9703aaf8a2e4, user=administrator, url=https://10.150.29.72/sdk, pass=*******} ran 56 seconds ago, consumed 22 seconds ago, 1min itv: collecting.

When the ESX host is actually removed from the original vSphere client (connected to the vCenter Server), they are removed from the HQ vSphere Inventory tree and no measurements are still scheduled.

Show
Frederic Calindas added a comment - Verified in 4.5.0-EE-143. When a ESX host is mvoed from one vCenter server to another, measurements are still trying to be collected as the Host/VMs are still present in HQ vSphere though they show in the inventory tree as gray and italics. In the vSphere client (connected to the vCenter Server), they show as disconnected. The agent.log of the vCenter server where the ESX host was moved from shows one of the VMs still collecting measurements: 2010-10-25 17:52:22,226 DEBUG [CollectorThread] [Collector] {vm=FC-TemplateTest-VM1, uuid=4206ed55-040d-1c00-780d-9703aaf8a2e4, user=administrator, url=https://10.150.29.72/sdk, pass=*******} ran 56 seconds ago, consumed 22 seconds ago, 1min itv: collecting. When the ESX host is actually removed from the original vSphere client (connected to the vCenter Server), they are removed from the HQ vSphere Inventory tree and no measurements are still scheduled.
Hide
Frederic Calindas added a comment -

Correction... even after removing the ESX host from the vCenter server, the agent is still reporting collecting measurements for VMs of the removed ESX host.

Show
Frederic Calindas added a comment - Correction... even after removing the ESX host from the vCenter server, the agent is still reporting collecting measurements for VMs of the removed ESX host.
Hide
Yoav Epelman added a comment -

Bulk change to new components

Show
Yoav Epelman added a comment - Bulk change to new components
Hide
Idan Hod added a comment -

As part of our continuous effort to improve product quality, The Hyperic product team has decided to implement a "zero bug policy" methodology.

Following this methodology, only defects that are planned to be handled in the near future will remain open. Any other defect will be deferred, with the option to be reevaluated if the need arises, or if changes to the Hyperic road-map make such defect a candidate for a fix.

We believe this new process will help create clarity and focus in the Hyperic road-map and ultimately benefit our customer base.

This bug has been deferred as part of the new policy.

We appreciate your cooperation and continues contribution to the improvement of Hyperic.

Show
Idan Hod added a comment - As part of our continuous effort to improve product quality, The Hyperic product team has decided to implement a "zero bug policy" methodology. Following this methodology, only defects that are planned to be handled in the near future will remain open. Any other defect will be deferred, with the option to be reevaluated if the need arises, or if changes to the Hyperic road-map make such defect a candidate for a fix. We believe this new process will help create clarity and focus in the Hyperic road-map and ultimately benefit our customer base. This bug has been deferred as part of the new policy. We appreciate your cooperation and continues contribution to the improvement of Hyperic.

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved:
    Last comment:
    41 weeks, 2 days ago