Deprecated: Function get_magic_quotes_gpc() is deprecated in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 99

Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 619

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1169

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176

Warning: Cannot modify header information - headers already sent by (output started at /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php:99) in /hermes/walnacweb04/walnacweb04ab/b2791/pow.jasaeld/htdocs/De1337/nothing/index.php on line 1176
10BC0 [WIP on-hold] Reimplement metric store in pure java by andsel · Pull Request #17890 · elastic/logstash · GitHub
Nothing Special   »   [go: up one dir, main page]

Skip to content
6880
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions logstash-core/lib/logstash/instrument/collector.rb
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,16 @@ def initialize
def push(namespaces_path, key, type, *metric_type_params)
begin
metric_proxy = get(namespaces_path, key, type)
return metric_proxy.execute(*metric_type_params) if metric_proxy.respond_to?(:execute)

logger.error("Collector: Cannot record metric action #{type}@#{metric_type_params.join('/')} on <#{metric_proxy}> at path #{namespaces_path.join('/')}/#{key}")
expended_params = *metric_type_params
case type
when :gauge
return metric_proxy.set(expended_params[1])
when :counter
return metric_proxy.increment if expended_params[1] == nil
return metric_proxy.increment(expended_params[1])
else
logger.error("Collector: Cannot record metric action #{type}@#{metric_type_params.join('/')} on <#{metric_proxy}> at path #{namespaces_path.join('/')}/#{key}")
end
rescue MetricStore::NamespacesExpectedError => e
logger.error("Collector: Cannot record metric", :exception => e)
rescue NameError => e
Expand Down
72 changes: 70 additions & 2 deletions logstash-core/lib/logstash/instrument/metric_store.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,10 @@ module LogStash module Instrument
# saved in a retrievable way, this is a wrapper around multiples ConcurrentHashMap
# acting as a tree like structure.
class MetricStore
class NamespacesExpectedError < StandardError; end
class MetricNotFound < StandardError; end
# class NamespacesExpectedError < StandardError; end
# class MetricNotFound < StandardError; end
java_import org.logstash.instrument.metrics.MetricStore::MetricNotFound
java_import org.logstash.instrument.metrics.MetricStore::NamespacesExpectedError

KEY_PATH_SEPARATOR = "/".freeze

Expand All @@ -47,6 +49,9 @@ def initialize
# in the structured hash or when we query it for search or to make
# the result available in the API.
@structured_lookup_mutex = Mutex.new

@java_store = org.logstash.instrument.metrics.MetricStore.new
@use_java_impl = true
end

# This method use the namespace and key to search the corresponding value of
Expand All @@ -66,6 +71,8 @@ def initialize
# to the provided default_value_generator block will be stored.
# @return [Metric] the value as it exists in the tree after this operation
def fetch_or_store(namespaces, key, default_value = nil)
return @java_store.fetch_or_store(namespaces.map(&:to_s), key.to_s, block_given? ? yield(key) : default_value) if @use_java_impl

# We first check in the `@fast_lookup` store to see if we have already see that metrics before,
# This give us a `o(1)` access, which is faster than searching through the structured
# data store (Which is a `o(n)` operation where `n` is the number of element in the namespace and
Expand Down Expand Up @@ -104,6 +111,9 @@ def fetch_or_store(namespaces, key, default_value = nil)
# @param [Array] The path where values should be located
# @return [Hash]
def get_with_path(path)
return remap_keys_to_sym(@java_store.get_with_path(path)) if @use_java_impl
# return @java_store.get_with_path(java.util.ArrayList.new(path.map(&:to_s))) if @use_java_impl

get(*key_paths(path))
end

Expand All @@ -112,6 +122,9 @@ def get_with_path(path)
# @param [Array<Symbol>]
# @return [Hash]
def get(*key_paths)
#TODO ? remap key strings to key symbols or the Java get method should work with RubySymbol-s?
return remap_keys_to_sym(@java_store.get(java.util.ArrayList.new(key_paths.map(&:to_s)))) if @use_java_impl

# Normalize the symbols access
key_paths.map(&:to_sym)
new_hash = Hash.new
Expand All @@ -123,13 +136,42 @@ def get(*key_paths)
new_hash
end

def remap_keys_to_sym(map)
translated_map = {}
map.each do |key, value|
if value.is_a?(Hash)
translated_map[key.to_sym] = remap_keys_to_sym(value)
else
translated_map[key.to_sym] = value
end
#map.delete( 5CC0 key)
end
translated_map
end
private :remap_keys_to_sym

def remap_nested_to_string(list)
result = java.util.ArrayList.new(list.size)
list.each do |v|
if v.is_a?(Array)
result << remap_nested_to_string(v)
else
result << v.to_s
end
end
result
end
private :remap_nested_to_string

# Retrieve values like `get`, but don't return them fully nested.
# This means that if you call `get_shallow(:foo, :bar)` the result will not
# be nested inside of `{:foo {:bar => values}`.
#
# @param [Array<Symbol>]
# @return [Hash]
def get_shallow(*key_paths)
return remap_keys_to_sym(@java_store.get_shallow(key_paths.map(&:to_s))) if @use_java_impl

key_paths.reduce(get(*key_paths)) {|acc, p| acc[p]}
end

Expand All @@ -154,6 +196,8 @@ def get_shallow(*key_paths)
# }
# }
def extract_metrics(path, *keys)
return remap_keys_to_sym(@java_store.extract_metrics(java.util.ArrayList.new(path.map(&:to_s)), remap_nested_to_string(keys))) if @use_java_impl

keys.reduce({}) do |acc, k|
# Simplify 1-length keys
k = k.first if k.is_a?(Array) && k.size == 1
Expand Down Expand Up @@ -187,6 +231,10 @@ def extract_metrics(path, *keys)
end

def has_metric?(*path)
if @use_java_impl
return @java_store.has_metric?(java.util.ArrayList.new(path.map(&:to_s)))
end

@fast_lookup[path]
end

Expand All @@ -196,6 +244,22 @@ def has_metric?(*path)
# @param path [String] The search path for metrics
# @param [Array] The metric for the specific path
def each(path = nil, &block)
if @use_java_impl
if path.nil?
if block_given?
return @java_store.each(&block)
else
return @java_store.each
end
else
if block_given?
return @java_store.each(path, &block)
else
return @java_store.each(path)
end
end
end

metrics = if path.nil?
get_all
else
Expand All @@ -207,6 +271,8 @@ def each(path = nil, &block)
alias_method :all, :each

def prune(path)
return @java_store.prune(path) if @use_java_impl

key_paths = key_paths(path).map(&:to_sym)
@structured_lookup_mutex.synchronize do
keys_to_delete = @fast_lookup.keys.select {|namespace| (key_paths - namespace[0..-2]).empty? }
Expand All @@ -216,6 +282,8 @@ def prune(path)
end

def size
return @java_store.size if @use_java_impl

@fast_lookup.size
end

Expand Down
7 changes: 2 additions & 5 deletions logstash-core/lib/logstash/instrument/metric_type.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@
# specific language governing permissions and limitations
# under the License.

require "logstash/instrument/metric_type/counter"
require "logstash/instrument/metric_type/gauge"

module LogStash module Instrument
module MetricType
# Use the string to generate a concrete class for this metrics
Expand All @@ -28,8 +25,8 @@ module MetricType
# @raise [NameError] If the class is not found
def self.create(type, namespaces, key)
case type
when :counter then return LogStash::Instrument::MetricType::Counter.new(namespaces, key)
when :gauge then return LogStash::Instrument::MetricType::Gauge.new(namespaces, key)
when :counter then return org.logstash.instrument.metrics.counter.LongCounter.new(key.to_s)
when :gauge then return org.logstash.instrument.metrics.gauge.LazyDelegatingGauge.new(key.to_s)
when :uptime then return org.logstash.instrument.metrics.UptimeMetric.new(key.to_s)
when :timer then return org.logstash.instrument.metrics.timer.TimerMetric::create(key.to_s)
else fail NameError, "Unknown Metric Type `#{type}`"
Expand Down
32 changes: 0 additions & 32 deletions logstash-core/lib/logstash/instrument/metric_type/counter.rb

This file was deleted.

31 changes: 0 additions & 31 deletions logstash-core/lib/logstash/instrument/metric_type/gauge.rb

This file was deleted.

4 changes: 2 additions & 2 deletions logstash-core/spec/logstash/instrument/collector_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@
let(:key) { :my_key }

{
counter: LogStash::Instrument::MetricType::Counter,
gauge: LogStash::Instrument::MetricType::Gauge,
counter: org.logstash.instrument.metrics.counter.LongCounter,
gauge: org.logstash.instrument.metrics.gauge.LazyDelegatingGauge,
uptime: org.logstash.instrument.metrics.UptimeMetric,
timer: org.logstash.instrument.metrics.timer.TimerMetric,
}.each do |type, type_specific_implementation|
Expand Down
Loading
Loading
0