Skip to content

Jules (adde excel + excel multi tab provider) #452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Whether you're tracking metrics, analyzing trends, or monitoring performance, th
## Key Features

- **Visualize Nested Data**: Display hierarchical data structures in pie charts, trend charts, and tables.
- **Multiple File Formats**: Supports JSON, YAML, XML, and CSV files.
- **Multiple File Formats**: Supports JSON, YAML, XML, CSV, and Excel (.xls, .xlsx) files.
- **Dynamic UI**: Interactive charts and tables that update based on your data.
- **Customizable Colors**: Define custom colors for your data points or use predefined color schemes.
- **Trend Analysis**: Track data trends over multiple builds with history charts.
Expand Down Expand Up @@ -85,6 +85,34 @@ The plugin supports the following file formats for data input:
#### YAML and XML
- Similar hierarchical structures as JSON are supported.

#### Excel (`excel` provider)
- This provider parses a single Excel sheet from an `.xls` or `.xlsx` file. By default, it processes the **first sheet** in the workbook.
- **Structure Expectation:**
- The parser automatically detects the header row (the first non-empty row).
- Columns *before* the first column containing predominantly numeric data are treated as hierarchy levels.
- Columns *from* the first numeric-looking column onwards are treated as data values, with their respective header names as keys for the results.
- **Example Data (conceptual view of a sheet):**
```
(Sheet1 in an .xlsx or .xls file)
Category, SubCategory, Metric1, Value2
Alpha, X, 10, 100
Alpha, Y, 15, 150
Beta, Z, 20, 200
```
In this example:
- "Category" and "SubCategory" would form the hierarchy (e.g., Alpha -> X).
- "Metric1" and "Value2" would be the data keys with their corresponding numeric values.
- Empty rows before the header or between data rows are typically ignored.

#### Multi-Sheet Excel (`excelmulti` provider)
- This provider parses **all sheets** in an Excel workbook (.xls or .xlsx).
- **Header Consistency Requirement:**
- The header from the *first successfully parsed sheet* (first non-empty sheet with a valid header) is used as a reference.
- Subsequent sheets **must have an identical header** (same column names in the same order) to be included in the report.
- Sheets with headers that do not match the reference header will be skipped, and a warning will be logged.
- **Data Structure per Sheet:** Within each sheet, the data structure expectation is the same as for the `excel` provider (auto-detected header, hierarchy based on pre-numeric columns, values from numeric columns onwards).
- Item IDs are generated to be unique across sheets, typically by internally prefixing them with sheet-specific information.

---

## Color Management
Expand All @@ -95,7 +123,7 @@ The plugin allows you to customize the colors used in the visualizations. You ca

To customize colors, add a `colors` object to your JSON, YAML, or XML file. The `colors` object should map metric keys or category names to specific colors. Colors can be defined using **HEX values** or **predefined color names**.

> **Note**: Color customization is **not supported for CSV files** due to the format does not allow color attribute definition. For now, colors are attributed aleatory.
> **Note**: Color customization is **not supported for CSV or Excel files** as these formats do not have a standard way to define color attributes within the data file itself for this plugin's use. For CSV and Excel, colors are attributed automatically by the charting libraries.

#### Example in JSON:
```json
Expand Down Expand Up @@ -146,9 +174,17 @@ You can interact with the charts and tables to drill down into specific data poi
- `relative`: Show percentage values.
- `dual`: Show both absolute and relative values.
- **`provider`**: Specify the file format and pattern for the data files.
- **`id`**: (Required for CSV) A unique identifier for the report.
- **`id`**: (Optional, but recommended for CSV, Excel, and ExcelMulti if multiple reports of the same type are used) A unique identifier for the report instance. This helps in creating distinct report URLs and managing history, especially if you have multiple CSV or Excel reports in the same job.
- **`pattern`**: An Ant-style pattern to locate the data files.

**Examples for `provider`:**
- JSON: `provider: json(pattern: 'reports/**/*.json')`
- CSV: `provider: csv(id: 'my-csv-report', pattern: 'reports/data.csv')`
- Excel (single sheet): `provider: excel(pattern: 'reports/data.xlsx')`
- Excel (multi-sheet): `provider: excelmulti(pattern: 'reports/multi_sheet_data.xlsx')`
- You can also add an `id` to `excel` and `excelmulti` if needed:
`provider: excel(id: 'my-excel-report', pattern: 'reports/data.xlsx')`


## Examples

Expand Down
19 changes: 19 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,25 @@
<artifactId>jackson2-api</artifactId>
</dependency>

<!-- Apache POI -->
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi</artifactId>
<version>5.4.1</version>
</dependency>
<dependency>
<groupId>org.apache.poi</groupId>
<artifactId>poi-ooxml</artifactId>
<version>5.4.1</version>
</dependency>

<!-- JSR 305 for annotations -->
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>3.0.2</version>
</dependency>

<!-- Workflow dependencies -->
<dependency>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package io.jenkins.plugins.reporter.model;

import java.io.Serializable;

public class ExcelParserConfig implements Serializable {

private static final long serialVersionUID = 1L;

// Future configuration options can be added here, for example:
// private int headerRowIndex = 0; // Default to the first row
// private int dataStartRowIndex = 1; // Default to the second row
// private String sheetName; // For single sheet parsing, if specified
// private boolean detectHeadersAutomatically = true;

public ExcelParserConfig() {
// Default constructor
}

// Add getters and setters here if fields are added in the future.
}
12 changes: 10 additions & 2 deletions src/main/java/io/jenkins/plugins/reporter/model/Item.java
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,14 @@ public LinkedHashMap<String, Integer> getResult() {
return result;
}

return getItems()
// NPE fix: check if items list is null or empty before streaming
if (items == null || items.isEmpty()) { // items is the List<Item> field
return new LinkedHashMap<>(); // Return empty map if no sub-items to aggregate from
}

return items // Now items is guaranteed not to be null and not empty
.stream()
.map(Item::getResult)
.map(Item::getResult) // Recursive call
.flatMap(map -> map.entrySet().stream())
.collect(Collectors.groupingBy(Map.Entry::getKey, LinkedHashMap::new, Collectors.summingInt(Map.Entry::getValue)));
}
Expand Down Expand Up @@ -114,6 +119,9 @@ public void setItems(List<Item> items) {
}

public void addItem(Item item) {
if (this.items == null) {
this.items = new ArrayList<>();
}
this.items.add(item);
}
}
12 changes: 12 additions & 0 deletions src/main/java/io/jenkins/plugins/reporter/model/ReportDto.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ public class ReportDto extends ReportBase {
@JsonInclude(JsonInclude.Include.NON_NULL)
private Map<String, String> colors;

@JsonProperty(value = "parserLogMessages")
@JsonInclude(JsonInclude.Include.NON_EMPTY) // Only include in JSON if not empty
private List<String> parserLogMessages;

public String getId() {
return id;
}
Expand All @@ -42,6 +46,14 @@ public Map<String, String> getColors() {
public void setColors(Map<String, String> colors) {
this.colors = colors;
}

public List<String> getParserLogMessages() {
return parserLogMessages;
}

public void setParserLogMessages(List<String> parserLogMessages) {
this.parserLogMessages = parserLogMessages;
}

@JsonIgnore
public Report toReport() {
Expand Down
Loading
Loading