Skip to content

Commit 0ff1907

Browse files
akshathskakshathsk
committed
Adding support for Apache Skywalking
1 parent a435231 commit 0ff1907

File tree

27 files changed

+4040
-0
lines changed

27 files changed

+4040
-0
lines changed

‎.gitignore

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,3 +127,29 @@ dmypy.json
127127

128128
# Pyre type checker
129129
.pyre/
130+
131+
.DS_Store
132+
133+
# Compiled class file
134+
*.class
135+
136+
# Log file
137+
*.log
138+
139+
# BlueJ files
140+
*.ctxt
141+
142+
# Mobile Tools for Java (J2ME)
143+
.mtj.tmp/
144+
145+
# Package Files #
146+
*.jar
147+
*.war
148+
*.nar
149+
*.ear
150+
*.zip
151+
*.tar.gz
152+
*.rar
153+
154+
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
155+
hs_err_pid*

‎core/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ In the container, you can generate and run ctests as shown below.
4848

4949
Run `./setup_ubuntu.sh` or `sudo ./setup_ubuntu.sh`.
5050

51+
For Apache Skywalking Project - Run `sudo ./setup_ubuntu_skywalking.sh`
52+
5153
### 2. Target Project
5254

5355
To generate ctests or run ctest, you need to first clone the target project.
@@ -61,6 +63,8 @@ To generate ctests or run ctest, you need to first clone the target project.
6163

6264
`<main project>` can be `hadoop`, `hbase`, `zookeeper` or `alluxio`.
6365

66+
For Apache Skywalking Project - Run `./add_project_skywalking.sh -r <skwywalking_release_version>`
67+
6468
## Usage
6569

6670
### 1. Generating Ctests

‎core/add_project_skywalking.sh

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
#!/bin/bash
2+
3+
while getopts r: flag
4+
do
5+
case "${flag}" in
6+
r) version=${OPTARG};;
7+
esac
8+
done
9+
10+
if [ -z "$version" ]
11+
then
12+
echo "Usage : ./add_project_skywalking.sh -r <release_version>"
13+
exit
14+
fi
15+
16+
echo $version
17+
18+
rm -rf app/skywalking
19+
20+
mkdir -p app/skywalking
21+
22+
cd app/skywalking
23+
24+
wget https://dlcdn.apache.org/skywalking/$version/apache-skywalking-apm-$version-src.tgz
25+
26+
tar zxvf apache-skywalking-apm-$version-src.tgz
27+
28+
cd apache-skywalking-apm-$version
29+
30+
git apply ../../../patch/skywalking/skywalking.patch
31+
32+
cd oap-server
33+
34+
mvn clean install -DskipTests
35+
36+
cd ..

‎core/default_configs/skywalking-default.tsv

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
configuration.apollo.apolloEnv DEV Configure appolo environment
2+
services[1].duration 20000 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
3+
configuration.nacos.port 49317 Nacos Server Port
4+
configuration.zookeeper.hostPort localhost:49315 Zookeeper host port
5+
configuration.nacos.username nacos Nacos auth username
6+
configuration.nacos.port 49319 Nacos Server Port
7+
storage.mysql.metadataQueryMaxSize 5000 Metadata query max size
8+
configuration.apollo.period 1 Unit seconds sync period default fetch every 60 seconds
9+
configuration.nacos.period 2 Unit seconds sync period default fetch every 60 seconds
10+
configuration.nacos.accessKey Nacos auth accessKey
11+
configuration.consul.period 1 Sync period in seconds. Defaults to 60 seconds.
12+
configuration.etcd.period 1 Unit seconds sync period default fetch every 60 seconds
13+
configuration.zookeeper.maxRetries 3 max number of times to retry
14+
configuration.apollo.apolloCluster default Apollo cluster name
15+
configuration.zookeeper.hostPort localhost:49310 Zookeeper host port
16+
core.default.enableDataKeeperExecutor TRUE Set a timeout on metrics data. After the timeout has expired, the metrics data will automatically be deleted.
17+
default.rate 10000 The sample rate precision is 1/10000. 10000 means 100% sample in default
18+
receiver-zipkin.default.restPort 9411 Port used by OAP to collect zipkin trace from HTTP
19+
services[1].name name1 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
20+
services[0].rate 2000 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
21+
MO-1.MO-1-P1.prop2 value2-prop Example configs
22+
MO-1.MO-1-P1.p1 value1 Example configs
23+
configuration.consul.hostAndPorts localhost:49296 Consul host and ports separated by comma eg 1.2.3.4:8500,2.3.4.5:8500
24+
configuration.zookeeper.namespace /default Zookeeper namespace
25+
configuration.nacos.secretKey Nacos auth accessKey
26+
configuration.apollo.appId SampleApp Apollo application id
27+
storage.mysql.properties.jdbcUrl jdbc:mysql://localhost:3306/swtest?rewriteBatchedStatements Mysql jdbc connection string
28+
configuration.nacos.serverAddr localhost Nacos Server Host
29+
configuration.etcd.endpoints http://127.0.0.1:49305 Etcd Server Host
30+
services[1].rate 1000 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
31+
configuration.nacos.group skywalking Nacos Configuration Group
32+
configuration.apollo.apolloMeta http://localhost:49284 Apollo host and port
33+
configuration.etcd.endpoints http://127.0.0.1:49302 Etcd Server Host
34+
services[0].name name2 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
35+
configuration.zookeeper.baseSleepTimeMs 1000 initial amount of time to wait between retries
36+
storage.mysql.properties.dataSource.password !AI!3B Mysql datasource password
37+
configuration.apollo.apolloMeta http://localhost:49287 Apollo host and port
38+
default.duration -1 The sample rate precision is 1/10000. 10000 means 100% sample in default
39+
configuration.consul.hostAndPorts localhost:49290 localhost:49296 Consul host and ports separated by comma eg 1.2.3.4:8500,2.3.4.5:8500
40+
receiver-zipkin.default.restHost 0.0.0.0 Host used by OAP to collect zipkin trace from HTTP
41+
configuration.nacos.namespace Nacos Configuration namespace
42+
configuration.nacos.password nacos Nacos auth username
43+
MO-1.MO-1-P1.p2 value2 Example configs
44+
services[0].duration 30000 The sample rate precision is 1/10000. 10000 means 100% sample in default. Setting this threshold about the latency would make the slow trace segments sampled if they cost more time, even the sampling mechanism activated. The default value is `-1`, which means would not sample slow traces. Unit, millisecond.
45+
configuration.zookeeper.period 1 Unit seconds sync period default fetch every 60 seconds
46+
configuration.etcd.namespace /skywalking/ Etcd Configuration Group

‎core/generate_ctest/skywalking/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Generating Parameter Sets for Apache Skywalking
2+
3+
**generate_ctest is used to automatically generate ctests**
4+
5+
```shell
6+
# Switch to the generate_ctest/skywalking directory
7+
cd <openctest dir>/core/generate_ctest/skywalking
8+
9+
# Build the project to generate the target/ctest-runner-1.0-SNAPSHOT-jar-with-dependencies.jar
10+
mvn clean package
11+
12+
# Switch to the instrumented apache skywalking directory
13+
cd /home/skywalking
14+
15+
# Run the jar by providing the following inputs
16+
# 1. temp directory which will contain the files needed for this run
17+
# 2. File test-to-config-flat.txt from the execution of the core/identify_param/skywalking step
18+
# 3. File skywalking-generated-values.tsv from the execution of the core/generate_value step
19+
# 4. Result directory which will contain the results of the execution
20+
java -jar <openctest dir>/core/generate_ctest/skywalking/target/ctest-runner-1.0-SNAPSHOT-jar-with-dependencies.jar <dir>/tmp <dir>/test-to-config-flat.txt <dir>/skywalking-generated-values.tsv <dir>/resultDir
21+
```
22+
23+
## Results
24+
25+
2 files are generated as a result of this run
26+
27+
1. Generates the `data/ctest_mapping/skywalking.json` file
28+
```json
29+
{
30+
"configuration.apollo.period": [
31+
"org.apache.skywalking.oap.server.configuration.apollo.ITApolloConfigurationTest#shouldReadUpdated",
32+
"org.apache.skywalking.oap.server.configuration.apollo.ITApolloConfigurationTest#shouldReadUpdated4Group"
33+
],
34+
"configuration.zookeeper.maxRetries": [
35+
"org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated",
36+
"org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated4GroupConfig"
37+
],
38+
"configuration.zookeeper.period": [
39+
"org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated",
40+
"org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated4GroupConfig"
41+
]...
42+
}
43+
44+
```
45+
46+
2. Execution results for each case in the tab separated format
47+
48+
Param Test Value p/f Time_in_ns
49+
```json
50+
services[0].duration org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testDefaultSampleRateNotify 15000 p 9188519779
51+
services[0].duration org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testDefaultSampleRateNotify 60000 p 9179170267
52+
```

‎core/generate_ctest/skywalking/pom.xml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<groupId>org.uiuc</groupId>
8+
<artifactId>ctest-runner</artifactId>
9+
<version>1.0-SNAPSHOT</version>
10+
11+
<properties>
12+
<maven.compiler.source>8</maven.compiler.source>
13+
<maven.compiler.target>8</maven.compiler.target>
14+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
15+
</properties>
16+
17+
<dependencies>
18+
<dependency>
19+
<groupId>org.apache.maven.shared</groupId>
20+
<artifactId>maven-invoker</artifactId>
21+
<version>3.2.0</version>
22+
</dependency>
23+
<dependency>
24+
<groupId>com.google.code.gson</groupId>
25+
<artifactId>gson</artifactId>
26+
<version>2.9.1</version>
27+
</dependency>
28+
<dependency>
29+
<groupId>com.anubhavshukla</groupId>
30+
<artifactId>properties-to-yaml-converter</artifactId>
31+
<version>1.0.1</version>
32+
</dependency>
33+
</dependencies>
34+
35+
<build>
36+
<plugins>
37+
<plugin>
38+
<artifactId>maven-assembly-plugin</artifactId>
39+
<configuration>
40+
<archive>
41+
<manifest>
42+
<mainClass>org.uiuc.Main</mainClass>
43+
</manifest>
44+
</archive>
45+
<descriptorRefs>
46+
<descriptorRef>jar-with-dependencies</descriptorRef>
47+
</descriptorRefs>
48+
</configuration>
49+
<executions>
50+
<execution>
51+
<id>make-assembly</id> <!-- this is used for inheritance merges -->
52+
<phase>package</phase> <!-- bind to the packaging phase -->
53+
<goals>
54+
<goal>single</goal>
55+
</goals>
56+
</execution>
57+
</executions>
58+
</plugin>
59+
</plugins>
60+
</build>
61+
62+
</project>

‎core/generate_ctest/skywalking/src/main/java/org/uiuc/AppConstants.java

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
package org.uiuc;
2+
3+
import java.util.Arrays;
4+
import java.util.HashMap;
5+
import java.util.List;
6+
import java.util.Map;
7+
8+
public interface AppConstants {
9+
10+
Map<String, String> moduleToDirMap = new HashMap<String, String>() {
11+
{
12+
put("oap-server/server-starter", "./oap-server/server-starter/src/main/resources/");
13+
put("oap-server/server-configuration/configuration-apollo", "./oap-server/server-configuration/configuration-apollo/src/test/resources/");
14+
put("oap-server/server-configuration/configuration-etcd", "./oap-server/server-configuration/configuration-etcd/src/test/resources/");
15+
put("oap-server/server-configuration/configuration-zookeeper", "./oap-server/server-configuration/configuration-zookeeper/src/test/resources/");
16+
put("oap-server/analyzer/agent-analyzer", "./oap-server/analyzer/agent-analyzer/src/main/resources/");
17+
put("oap-server/server-configuration/configuration-nacos", "./oap-server/server-configuration/configuration-nacos/src/test/resources/");
18+
put("oap-server/server-configuration/configuration-consul", "./oap-server/server-configuration/configuration-consul/src/test/resources/");
19+
}
20+
};
21+
22+
Map<String, String> moduleToFileNameMap = new HashMap<String, String>() {
23+
{
24+
put("oap-server/server-starter", "application2.yml");
25+
put("oap-server/server-configuration/configuration-apollo", "application2.yml");
26+
put("oap-server/server-configuration/configuration-etcd", "application2.yml");
27+
put("oap-server/server-configuration/configuration-zookeeper", "application2.yml");
28+
put("oap-server/analyzer/agent-analyzer", "trace-sampling-policy-settings2.yml");
29+
put("oap-server/server-configuration/configuration-nacos", "application2.yml");
30+
put("oap-server/server-configuration/configuration-consul", "application2.yml");
31+
}
32+
};
33+
34+
List<String> testCases = Arrays.asList(
35+
"oap-server/server-starter>org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoaderTestCase#testLoadConfig",
36+
"oap-server/server-starter>org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoaderTestCase#testLoadStringTypeConfig",
37+
"oap-server/server-starter>org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoaderTestCase#testLoadIntegerTypeConfig",
38+
"oap-server/server-starter>org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoaderTestCase#testLoadBooleanTypeConfig",
39+
"oap-server/server-starter>org.apache.skywalking.oap.server.starter.config.ApplicationConfigLoaderTestCase#testLoadSpecialStringTypeConfig",
40+
"oap-server/server-library/library-module>org.apache.skywalking.oap.server.library.module.ApplicationConfigurationTest#testBuildConfig",
41+
"oap-server/server-configuration/configuration-apollo>org.apache.skywalking.oap.server.configuration.apollo.ITApolloConfigurationTest#shouldReadUpdated",
42+
"oap-server/server-configuration/configuration-apollo>org.apache.skywalking.oap.server.configuration.apollo.ITApolloConfigurationTest#shouldReadUpdated4Group",
43+
"oap-server/server-configuration/configuration-consul>org.apache.skywalking.oap.server.configuration.consul.ITConsulConfigurationTest#shouldReadUpdated",
44+
"oap-server/server-configuration/configuration-consul>org.apache.skywalking.oap.server.configuration.consul.ITConsulConfigurationTest#shouldReadUpdated4Group",
45+
"oap-server/server-configuration/configuration-etcd>org.apache.skywalking.oap.server.configuration.etcd.ITEtcdConfigurationTest#shouldReadUpdated",
46+
"oap-server/server-configuration/configuration-etcd>org.apache.skywalking.oap.server.configuration.etcd.ITEtcdConfigurationTest#shouldReadUpdated4Group",
47+
"oap-server/server-configuration/configuration-zookeeper>org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated",
48+
"oap-server/server-configuration/configuration-zookeeper>org.apache.skywalking.oap.server.configuration.zookeeper.it.ITZookeeperConfigurationTest#shouldReadUpdated4GroupConfig",
49+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testStaticConfigInit",
50+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testTraceLatencyThresholdDynamicUpdate",
51+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testTraceLatencyThresholdNotify",
52+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testDefaultSampleRateDynamicUpdate",
53+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testDefaultSampleRateNotify",
54+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testServiceSampleRateDynamicUpdate",
55+
"oap-server/analyzer/agent-analyzer>org.apache.skywalking.oap.server.analyzer.provider.trace.TraceSamplingPolicyWatcherTest#testServiceSampleRateNotify",
56+
"oap-server/server-configuration/configuration-nacos>org.apache.skywalking.oap.server.configuration.nacos.ITNacosConfigurationTest#shouldReadUpdated",
57+
"oap-server/server-configuration/configuration-nacos>org.apache.skywalking.oap.server.configuration.nacos.ITNacosConfigurationTest#shouldReadUpdatedGroup",
58+
"oap-server/server-configuration/configuration-nacos>org.apache.skywalking.oap.server.configuration.nacos.NacosConfigWatcherRegisterTest#shouldReadConfigs"
59+
);
60+
61+
String ERROR_MSG = "Error on exec() method";
62+
}

0 commit comments

Comments
 (0)