Skip to content

Commit 3b62690

Browse files
karenzonekarenzone
authored
Doc: Add input-agent doc file (#411)
1 parent de2f1b8 commit 3b62690

File tree

1 file changed

+320
-0
lines changed

1 file changed

+320
-0
lines changed

‎docs/agent.asciidoc‎

Lines changed: 320 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,320 @@
1+
:plugin: agent
2+
:type: input
3+
:default_codec: plain
4+
5+
///////////////////////////////////////////
6+
START - GENERATED VARIABLES, DO NOT EDIT!
7+
///////////////////////////////////////////
8+
9+
// Copied from Beats generated plugin output.
10+
// Not actively generated at this time!
11+
12+
////
13+
:version: %VERSION%
14+
:release_date: %RELEASE_DATE%
15+
:changelog_url: %CHANGELOG_URL%
16+
:include_path: ../../../../logstash/docs/include
17+
////
18+
19+
///////////////////////////////////////////
20+
END - GENERATED VARIABLES, DO NOT EDIT!
21+
///////////////////////////////////////////
22+
23+
[id="plugins-{type}s-{plugin}"]
24+
25+
=== Agent input plugin
26+
27+
include::{include_path}/plugin_header.asciidoc[]
28+
29+
==== Description
30+
31+
This input plugin enables Logstash to receive events from the
32+
https://www.elastic.co/products/beats[Elastic Beats] framework.
33+
34+
The following example shows how to configure Logstash to listen on port
35+
5044 for incoming Beats connections and to index into Elasticsearch.
36+
37+
[source,logstash]
38+
-----
39+
40+
input {
41+
beats {
42+
port => 5044
43+
}
44+
}
45+
46+
output {
47+
elasticsearch {
48+
hosts => ["http://localhost:9200"]
49+
index => "%{[@metadata][beat]}-%{[@metadata][version]}" <1>
50+
}
51+
}
52+
-----
53+
<1> `%{[@metadata][beat]}` sets the first part of the index name to the value
54+
of the `beat` metadata field and `%{[@metadata][version]}` sets the second part to
55+
the Beat's version. For example:
56+
metricbeat-7.4.0.
57+
58+
Events indexed into Elasticsearch with the Logstash configuration shown here
59+
will be similar to events directly indexed by Beats into Elasticsearch.
60+
61+
NOTE: If ILM is not being used, set `index` to
62+
`%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}` instead so
63+
Logstash creates an index per day, based on the `@timestamp` value of the events
64+
coming from Beats.
65+
66+
IMPORTANT: If you are shipping events that span multiple lines, you need to use
67+
the {filebeat-ref}/multiline-examples.html[configuration options available in
68+
Filebeat] to handle multiline events before sending the event data to Logstash.
69+
You cannot use the {logstash-ref}/plugins-codecs-multiline.html[Multiline codec
70+
plugin] to handle multiline events. Doing so will result in the failure to start
71+
Logstash.
72+
73+
[id="plugins-{type}s-{plugin}-versioned-indexes"]
74+
==== Versioned Beats Indices
75+
76+
To minimize the impact of future schema changes on your existing indices and
77+
mappings in Elasticsearch, configure the Elasticsearch output to write to
78+
versioned indices. The pattern that you specify for the `index` setting
79+
controls the index name:
80+
81+
[source,yaml]
82+
----
83+
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
84+
----
85+
86+
`%{[@metadata][beat]}`:: Sets the first part of the index name to the value of
87+
the `beat` metadata field, for example, `filebeat`.
88+
`%{[@metadata][version]}`:: Sets the second part of the name to the Beat
89+
version, for example, +{logstash_version}+.
90+
`%{+YYYY.MM.dd}`:: Sets the third part of the name to a date based on the
91+
Logstash `@timestamp` field.
92+
93+
This configuration results in daily index names like
94+
+filebeat-{logstash_version}-{localdate}+.
95+
96+
97+
[id="plugins-{type}s-{plugin}-ecs_metadata"]
98+
==== Event Metadata and the Elastic Common Schema (ECS)
99+
When decoding `beats` events, this plugin adds two fields related to the event: the deprecated `host`
100+
which contains the `hostname` provided by beats and the `ip_address` containing the remote address
101+
of the client's connection. When <<plugins-{type}s-{plugin}-ecs_compatibility,ECS compatibility mode>> is
102+
enabled these are now moved in ECS compatible namespace.
103+
104+
[id="plugins-{type}s-{plugin}-options"]
105+
==== Agent Input Configuration Options
106+
107+
This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
108+
109+
[cols="<,<,<",options="header",]
110+
|=======================================================================
111+
|Setting |Input type|Required
112+
| <<plugins-{type}s-{plugin}-add_hostname>> |<<boolean,boolean>>|No
113+
| <<plugins-{type}s-{plugin}-cipher_suites>> |<<array,array>>|No
114+
| <<plugins-{type}s-{plugin}-client_inactivity_timeout>> |<<number,number>>|No
115+
| <<plugins-{type}s-{plugin}-ecs_compatibility>> | <<string,string>>|No
116+
| <<plugins-{type}s-{plugin}-host>> |<<string,string>>|No
117+
| <<plugins-{type}s-{plugin}-include_codec_tag>> |<<boolean,boolean>>|No
118+
| <<plugins-{type}s-{plugin}-port>> |<<number,number>>|Yes
119+
| <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|No
120+
| <<plugins-{type}s-{plugin}-ssl_certificate>> |a valid filesystem path|No
121+
| <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> |<<array,array>>|No
122+
| <<plugins-{type}s-{plugin}-ssl_handshake_timeout>> |<<number,number>>|No
123+
| <<plugins-{type}s-{plugin}-ssl_key>> |a valid filesystem path|No
124+
| <<plugins-{type}s-{plugin}-ssl_key_passphrase>> |<<password,password>>|No
125+
| <<plugins-{type}s-{plugin}-ssl_verify_mode>> |<<string,string>>, one of `["none", "peer", "force_peer"]`|No
126+
| <<plugins-{type}s-{plugin}-ssl_peer_metadata>> |<<boolean,boolean>>|No
127+
| <<plugins-{type}s-{plugin}-tls_max_version>> |<<number,number>>|No
128+
| <<plugins-{type}s-{plugin}-tls_min_version>> |<<number,number>>|No
129+
|=======================================================================
130+
131+
Also see <<plugins-{type}s-{plugin}-common-options>> for a list of options supported by all
132+
input plugins.
133+
134+
&nbsp;
135+
136+
[id="plugins-{type}s-{plugin}-add_hostname"]
137+
===== `add_hostname`
138+
139+
deprecated[6.0.0, The default value has been changed to `false`. In 7.0.0 this setting will be removed]
140+
141+
* Value type is <<boolean,boolean>>
142+
* Default value is `false`
143+
144+
Flag to determine whether to add `host` field to event using the value supplied by the beat in the `hostname` field.
145+
146+
147+
[id="plugins-{type}s-{plugin}-cipher_suites"]
148+
===== `cipher_suites`
149+
150+
* Value type is <<array,array>>
151+
* Default value is `java.lang.String[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256]@459cfcca`
152+
153+
The list of ciphers suite to use, listed by priorities.
154+
155+
[id="plugins-{type}s-{plugin}-client_inactivity_timeout"]
156+
===== `client_inactivity_timeout`
157+
158+
* Value type is <<number,number>>
159+
* Default value is `60`
160+
161+
Close Idle clients after X seconds of inactivity.
162+
163+
[id="plugins-{type}s-{plugin}-ecs_compatibility"]
164+
===== `ecs_compatibility`
165+
166+
* Value type is <<string,string>>
167+
* Supported values are:
168+
** `disabled`: unstructured connection metadata added at root level
169+
** `v1`: structured connection metadata added under ECS compliant namespaces
170+
* Default value depends on which version of Logstash is running:
171+
** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
172+
** Otherwise, the default value is `disabled`.
173+
174+
Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)].
175+
The value of this setting affects the keys for the Beats connection's metadata on the event:
176+
177+
.Metadata Location by `ecs_compatibility` value
178+
[cols="<l,<l,e,<e"]
179+
|=======================================================================
180+
|`disabled` |`v1` |Availability |Description
181+
182+
|[host] |[@metadata][input][beats][host][name] |Always |Name or address of the beat host
183+
|[@metadata][ip_address] |[@metadata][input][beats][host][ip] |Always |IP address of the Beats client
184+
|[@metadata][tls_peer][status] | [@metadata][tls_peer][status] | When SSL related fields are populated | Contains "verified"/"unverified" labels in `disabled`, `true`/`false` in `v1`
185+
|[@metadata][tls_peer][protocol] | [@metadata][input][beats][tls][version_protocol] | When SSL status is "verified" | Contains the TLS version used (e.g. `TLSv1.2`)
186+
|[@metadata][tls_peer][subject] | [@metadata][input][beats][tls][client][subject] | When SSL status is "verified" | Contains the identity name of the remote end (e.g. `CN=artifacts-no-kpi.elastic.co`)
187+
|[@metadata][tls_peer][cipher_suite] | [@metadata][input][beats][tls][cipher] | When SSL status is "verified" | Contains the name of cipher suite used (e.g. `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256`)
188+
|=======================================================================
189+
190+
[id="plugins-{type}s-{plugin}-host"]
191+
===== `host`
192+
193+
* Value type is <<string,string>>
194+
* Default value is `"0.0.0.0"`
195+
196+
The IP address to listen on.
197+
198+
[id="plugins-{type}s-{plugin}-include_codec_tag"]
199+
===== `include_codec_tag`
200+
201+
* Value type is <<boolean,boolean>>
202+
* Default value is `true`
203+
204+
205+
206+
[id="plugins-{type}s-{plugin}-port"]
207+
===== `port`
208+
209+
* This is a required setting.
210+
* Value type is <<number,number>>
211+
* There is no default value for this setting.
212+
213+
The port to listen on.
214+
215+
[id="plugins-{type}s-{plugin}-ssl"]
216+
===== `ssl`
217+
218+
* Value type is <<boolean,boolean>>
219+
* Default value is `false`
220+
221+
Events are by default sent in plain text. You can
222+
enable encryption by setting `ssl` to true and configuring
223+
the `ssl_certificate` and `ssl_key` options.
224+
225+
[id="plugins-{type}s-{plugin}-ssl_certificate"]
226+
===== `ssl_certificate`
227+
228+
* Value type is <<path,path>>
229+
* There is no default value for this setting.
230+
231+
SSL certificate to use.
232+
233+
[id="plugins-{type}s-{plugin}-ssl_certificate_authorities"]
234+
===== `ssl_certificate_authorities`
235+
236+
* Value type is <<array,array>>
237+
* Default value is `[]`
238+
239+
Validate client certificates against these authorities.
240+
You can define multiple files or paths. All the certificates will
241+
be read and added to the trust store. You need to configure the `ssl_verify_mode`
242+
to `peer` or `force_peer` to enable the verification.
243+
244+
245+
[id="plugins-{type}s-{plugin}-ssl_handshake_timeout"]
246+
===== `ssl_handshake_timeout`
247+
248+
* Value type is <<number,number>>
249+
* Default value is `10000`
250+
251+
Time in milliseconds for an incomplete ssl handshake to timeout
252+
253+
[id="plugins-{type}s-{plugin}-ssl_key"]
254+
===== `ssl_key`
255+
256+
* Value type is <<path,path>>
257+
* There is no default value for this setting.
258+
259+
SSL key to use.
260+
NOTE: This key need to be in the PKCS8 format, you can convert it with https://www.openssl.org/docs/man1.1.0/apps/pkcs8.html[OpenSSL]
261+
for more information.
262+
263+
[id="plugins-{type}s-{plugin}-ssl_key_passphrase"]
264+
===== `ssl_key_passphrase`
265+
266+
* Value type is <<password,password>>
267+
* There is no default value for this setting.
268+
269+
SSL key passphrase to use.
270+
271+
[id="plugins-{type}s-{plugin}-ssl_verify_mode"]
272+
===== `ssl_verify_mode`
273+
274+
* Value can be any of: `none`, `peer`, `force_peer`
275+
* Default value is `"none"`
276+
277+
By default the server doesn't do any client verification.
278+
279+
`peer` will make the server ask the client to provide a certificate.
280+
If the client provides a certificate, it will be validated.
281+
282+
`force_peer` will make the server ask the client to provide a certificate.
283+
If the client doesn't provide a certificate, the connection will be closed.
284+
285+
This option needs to be used with `ssl_certificate_authorities` and a defined list of CAs.
286+
287+
[id="plugins-{type}s-{plugin}-ssl_peer_metadata"]
288+
===== `ssl_peer_metadata`
289+
290+
* Value type is <<boolean,boolean>>
291+
* Default value is `false`
292+
293+
Enables storing client certificate information in event's metadata.
294+
295+
This option is only valid when `ssl_verify_mode` is set to `peer` or `force_peer`.
296+
297+
[id="plugins-{type}s-{plugin}-tls_max_version"]
298+
===== `tls_max_version`
299+
300+
* Value type is <<number,number>>
301+
* Default value is `1.2`
302+
303+
The maximum TLS version allowed for the encrypted connections. The value must be the one of the following:
304+
1.0 for TLS 1.0, 1.1 for TLS 1.1, 1.2 for TLS 1.2
305+
306+
[id="plugins-{type}s-{plugin}-tls_min_version"]
307+
===== `tls_min_version`
308+
309+
* Value type is <<number,number>>
310+
* Default value is `1`
311+
312+
The minimum TLS version allowed for the encrypted connections. The value must be one of the following:
313+
1.0 for TLS 1.0, 1.1 for TLS 1.1, 1.2 for TLS 1.2
314+
315+
316+
317+
[id="plugins-{type}s-{plugin}-common-options"]
318+
include::{include_path}/{type}.asciidoc[]
319+
320+
:default_codec!:

0 commit comments

Comments
 (0)