Skip to content

Commit 665b0ba

Browse files
authored
Orb 768 mute tests sink error and idle (orb-community#2355)
* mute tests that validate sink status because still behave badly
1 parent 7046386 commit 665b0ba

2 files changed

Lines changed: 33 additions & 31 deletions

File tree

python-test/features/integration.feature

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -272,8 +272,8 @@ Scenario: Provision agent with tag matching existing group with multiple policie
272272
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
273273
And referred sink must have active state on response within 120 seconds
274274
And 20 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
275-
276-
@sanity @sink_status_idle
275+
@MUTE
276+
#@sanity @sink_status_idle
277277
Scenario: Sink idle after 5 minutes without metrics flow
278278
Given the Orb user has a registered account
279279
And the Orb user logs in
@@ -291,7 +291,8 @@ Scenario: Sink idle after 5 minutes without metrics flow
291291
When stop the orb-agent container
292292
Then referred sink must have idle state on response after 660 seconds
293293

294-
@sanity @sink_status_error
294+
@MUTE
295+
#@sanity @sink_status_error
295296
Scenario: Sink with invalid endpoint
296297
Given the Orb user has a registered account
297298
And the Orb user logs in
@@ -371,8 +372,8 @@ Scenario: Unapplying policies that failed by removing group
371372
And no dataset should be linked to the removed group anymore
372373
And 0 dataset(s) have validity valid and 4 have validity invalid in 30 seconds
373374

374-
375-
@smoke @sink_status_error
375+
@MUTE
376+
#@smoke @sink_status_error
376377
Scenario: Sink with invalid username
377378
Given the Orb user has a registered account
378379
And the Orb user logs in
@@ -391,8 +392,8 @@ Scenario: Sink with invalid username
391392
And referred sink must have error state on response within 120 seconds
392393
And 4 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
393394

394-
395-
@smoke @sink_status_error
395+
@MUTE
396+
#@sanity @sink_status_error
396397
Scenario: Sink with invalid password
397398
Given the Orb user has a registered account
398399
And the Orb user logs in
@@ -944,7 +945,8 @@ Scenario: Remove agent (check dataset)
944945
And last container created is exited after 120 seconds
945946
And 2 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
946947

947-
@smoke @sink_status_error
948+
@MUTE
949+
#@sanity @sink_status_error
948950
Scenario: Edit sink active and use invalid remote host
949951
Given the Orb user has a registered account
950952
And the Orb user logs in
@@ -964,8 +966,8 @@ Scenario: Edit sink active and use invalid remote host
964966
Then referred sink must have error state on response within 120 seconds
965967
And 10 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
966968

967-
968-
@smoke @sink_status_error
969+
@MUTE
970+
#@sanity @sink_status_error
969971
Scenario: Edit sink active and use invalid username
970972
Given the Orb user has a registered account
971973
And the Orb user logs in
@@ -985,8 +987,8 @@ Scenario: Edit sink active and use invalid username
985987
Then referred sink must have error state on response within 120 seconds
986988
And 10 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
987989

988-
989-
@smoke @sink_status_error
990+
@MUTE
991+
#@sanity @sink_status_error
990992
Scenario: Edit sink active and use invalid password
991993
Given the Orb user has a registered account
992994
And the Orb user logs in
@@ -1006,8 +1008,8 @@ Scenario: Edit sink active and use invalid password
10061008
Then referred sink must have error state on response within 120 seconds
10071009
And 10 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
10081010

1009-
1010-
@sanity @sink_status_error
1011+
@MUTE
1012+
#@sanity @sink_status_error
10111013
Scenario: Edit sink with invalid username and use valid one
10121014
Given the Orb user has a registered account
10131015
And the Orb user logs in
@@ -1028,8 +1030,8 @@ Scenario: Edit sink with invalid username and use valid one
10281030
Then referred sink must have active state on response within 120 seconds
10291031
And 4 dataset(s) have validity valid and 0 have validity invalid in 30 seconds
10301032

1031-
1032-
@sanity @sink_status_error
1033+
@MUTE
1034+
#@sanity @sink_status_error
10331035
Scenario: Edit sink with password and use valid one
10341036
Given the Orb user has a registered account
10351037
And the Orb user logs in
@@ -1135,13 +1137,13 @@ Scenario: Partial Update: sink status after updating only sink name
11351137
And this agent's heartbeat shows that 2 policies are applied and all has status running
11361138
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
11371139
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1138-
And referred sink must have active state on response within 120 seconds
1140+
And referred sink must have active state on response within 360 seconds
11391141
When the name of this sink is updated
11401142
Then the name updates to the new value and other fields remains the same
11411143
And referred sink must have active state on response after 360 seconds
11421144

11431145

1144-
@sanity @sink_partial_update
1146+
@sanity @sink_partial_update
11451147
Scenario: Partial Update: sink status after updating only sink description
11461148
Given the Orb user has a registered account
11471149
And the Orb user logs in
@@ -1154,7 +1156,7 @@ Scenario: Partial Update: sink status after updating only sink description
11541156
And this agent's heartbeat shows that 2 policies are applied and all has status running
11551157
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
11561158
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1157-
And referred sink must have active state on response within 120 seconds
1159+
And referred sink must have active state on response within 360 seconds
11581160
When the description of this sink is updated
11591161
Then the description updates to the new value and other fields remains the same
11601162
And referred sink must have active state on response after 360 seconds
@@ -1173,7 +1175,7 @@ Scenario: Partial Update: sink status after updating only sink tags
11731175
And this agent's heartbeat shows that 2 policies are applied and all has status running
11741176
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
11751177
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1176-
And referred sink must have active state on response within 120 seconds
1178+
And referred sink must have active state on response within 360 seconds
11771179
When the tags of this sink is updated
11781180
Then the tags updates to the new value and other fields remains the same
11791181
And referred sink must have active state on response after 360 seconds
@@ -1191,7 +1193,7 @@ Scenario: Partial Update: sink status after updating only sink configs
11911193
And 3 simple policies are applied to the group
11921194
And this agent's heartbeat shows that 3 policies are applied and all has status running
11931195
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1194-
And referred sink must have error state on response within 120 seconds
1196+
And referred sink must have error state on response within 360 seconds
11951197
When the config of this sink is updated
11961198
Then the config updates to the new value and other fields remains the same
11971199
And referred sink must have active state on response within 360 seconds
@@ -1210,7 +1212,7 @@ Scenario: Partial Update: sink status after updating only sink name and descript
12101212
And this agent's heartbeat shows that 2 policies are applied and all has status running
12111213
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
12121214
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1213-
And referred sink must have active state on response within 120 seconds
1215+
And referred sink must have active state on response within 360 seconds
12141216
When the name and description of this sink is updated
12151217
Then the name and description updates to the new value and other fields remains the same
12161218
And referred sink must have active state on response after 360 seconds
@@ -1228,7 +1230,7 @@ Scenario: Partial Update: sink status after updating only sink name and configs
12281230
And 3 simple policies are applied to the group
12291231
And this agent's heartbeat shows that 3 policies are applied and all has status running
12301232
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1231-
And referred sink must have error state on response within 120 seconds
1233+
And referred sink must have error state on response within 360 seconds
12321234
When the name and config of this sink is updated
12331235
Then the name and config updates to the new value and other fields remains the same
12341236
And referred sink must have active state on response within 360 seconds
@@ -1247,7 +1249,7 @@ Scenario: Partial Update: sink status after updating only sink name and tags
12471249
And this agent's heartbeat shows that 2 policies are applied and all has status running
12481250
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
12491251
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1250-
And referred sink must have active state on response within 120 seconds
1252+
And referred sink must have active state on response within 360 seconds
12511253
When the name and tags of this sink is updated
12521254
Then the name and tags updates to the new value and other fields remains the same
12531255
And referred sink must have active state on response after 360 seconds
@@ -1266,7 +1268,7 @@ Scenario: Partial Update: sink status after updating only sink description and t
12661268
And this agent's heartbeat shows that 2 policies are applied and all has status running
12671269
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
12681270
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1269-
And referred sink must have active state on response within 120 seconds
1271+
And referred sink must have active state on response within 360 seconds
12701272
When the description and tags of this sink is updated
12711273
Then the description and tags updates to the new value and other fields remains the same
12721274
And referred sink must have active state on response after 360 seconds
@@ -1302,7 +1304,7 @@ Scenario: Partial Update: sink status after updating only sink tags and configs
13021304
And 3 simple policies are applied to the group
13031305
And this agent's heartbeat shows that 3 policies are applied and all has status running
13041306
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1305-
And referred sink must have error state on response within 120 seconds
1307+
And referred sink must have error state on response within 360 seconds
13061308
When the tags and config of this sink is updated
13071309
Then the tags and config updates to the new value and other fields remains the same
13081310
And referred sink must have active state on response within 360 seconds
@@ -1321,7 +1323,7 @@ Scenario: Partial Update: sink status after updating only sink name, description
13211323
And this agent's heartbeat shows that 2 policies are applied and all has status running
13221324
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
13231325
And the container logs that were output after all policies have been applied contain the message "scraped metrics for policy" referred to each applied policy within 180 seconds
1324-
And referred sink must have active state on response within 120 seconds
1326+
And referred sink must have active state on response within 360 seconds
13251327
When the name, description and tags of this sink is updated
13261328
Then the name, description and tags updates to the new value and other fields remains the same
13271329
And referred sink must have active state on response after 360 seconds
@@ -1340,7 +1342,7 @@ Scenario: Partial Update: sink status after updating only sink name, description
13401342
And 3 simple policies are applied to the group
13411343
And this agent's heartbeat shows that 3 policies are applied and all has status running
13421344
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1343-
And referred sink must have error state on response within 120 seconds
1345+
And referred sink must have error state on response within 360 seconds
13441346
When the name, description and config of this sink is updated
13451347
Then the name, description and config updates to the new value and other fields remains the same
13461348
And referred sink must have active state on response within 360 seconds
@@ -1358,7 +1360,7 @@ Scenario: Partial Update: sink status after updating only sink name, tags and co
13581360
And 3 simple policies are applied to the group
13591361
And this agent's heartbeat shows that 3 policies are applied and all has status running
13601362
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1361-
And referred sink must have error state on response within 120 seconds
1363+
And referred sink must have error state on response within 360 seconds
13621364
When the name, tags and config of this sink is updated
13631365
Then the name, tags and config updates to the new value and other fields remains the same
13641366
And referred sink must have active state on response within 360 seconds
@@ -1376,7 +1378,7 @@ Scenario: Partial Update: sink status after updating only sink description, tags
13761378
And 3 simple policies are applied to the group
13771379
And this agent's heartbeat shows that 3 policies are applied and all has status running
13781380
And the container logs contain the message "policy applied successfully" referred to each policy within 30 seconds
1379-
And referred sink must have error state on response within 120 seconds
1381+
And referred sink must have error state on response within 360 seconds
13801382
When the description, tags and config of this sink is updated
13811383
Then the description, tags and config updates to the new value and other fields remains the same
13821384
And referred sink must have active state on response within 360 seconds

python-test/features/sinks.feature

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ Scenario: Partial Update: updating only sink name, description and tags
141141
Then the name, description and tags updates to the new value and other fields remains the same
142142

143143

144-
@sanity@sink_partial_update
144+
@sanity @sink_partial_update
145145
Scenario: Partial Update: updating only sink name, description and configs
146146
Given that the user has the prometheus/grafana credentials
147147
And the Orb user has a registered account

0 commit comments

Comments
 (0)