Monitoring API enhancements/cleanups for 5.0

[acchen97]

This issue will track the final iteration of monitoring API fixes/additions in preparation for beta1.

• Node Stats
  • Add duration_in_millis metric under /_node/stats/pipeline/events Metrics: track event time spent through entire pipeline #5731
  • Add JVM GC stats under /_node/stats/jvm Metrics: Add JVM GC stats #5730
  • Remove the /_node/mem resource as it's been properly moved under /_node/jvm/mem add changes made to monitoring APIs for alpha5 #5701 (comment)
  • Add config reload stats under new resource type /_node/stats/x Stats: Report config reload data #5680 (@jsvd)
• Node Info
  • Add config reload boolean to /_node/pipeline Have /_node/ show if Logstash is configured to automatically reload configs #5681
  • Add JVM GC strategy info under /_node/jvm Create GC collector info in monitoring API #5755
• Hot Threads
  • Ensure ?human option works correctly for hot_threads API. Additionally, the output for ?human=true appeared to be more verbose than the default format. Need to confirm scope of changes following alpha5 release. add changes made to monitoring APIs for alpha5 #5701 (comment)

[ph]

@alvin I've looked at the hot thread human format and the JSON one, and I can say that they are currently the same. I've created a PR to fixes some small issue with it and adding a test to make sure the thread count is taken into consideration in the case.

[acchen97]

@ph thanks, I'll spin up alpha5 today for a quick fact check here as well.

[acchen97]

@ph doing some testing against alpha5. A few notes below, but will update the initial issue accordingly.

• Hot threads appears mostly fixed, the ?human=false appears to still show the human format though.
• The "mem" stats are still showing up in the /_node top level, this (along with the /_node/mem resource) should be removed as its now properly under /_node/stats/jvm
• For adding basic stats [in/out/duration_in_millis] to specific input plugins, created a new issue (Add basic input plugin stats to monitoring API #5769) to track the 12 plugin scope. @suyograo maybe divide/conquer makes sense?
• For additional plugin level stats, let's add a few more things for Beats input and ES output. This issue (Initial Plugin Metrics #5428) can be used as reference, but I'll let you run with the scoping based on your discretion of importance vs size of work.

[ph]

@acchen97 the human=false is fixed in #5766 that I've made today.

[ph]

@acchen97 Updated with the latest merges

[acchen97]

Awesome, great progress @ph @jsvd

[suyograo]

Closing this -- we can move to outside of this meta issue

• Plugin Stats
  • Add basic input plugin stats (in/out/duration_in_millis) for initial 12 plugins Add basic input plugin stats to monitoring API #5769
  • Add more plugin level stats for Beats input and Elasticsearch output Initial Plugin Metrics #5428 [meta] Adding metrics support for logstash plugins #5717

[acchen97]

Moved these to a separate issue #5921.