Skip to content

Commit 51eba00

Browse files
authored
Merge pull request #293 from nf-core/dev
Dev -> Master for v2.0.0 release
2 parents 6924b66 + fd95002 commit 51eba00

File tree

200 files changed

+22118
-6165
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

200 files changed

+22118
-6165
lines changed

.editorconfig

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
root = true
2+
3+
[*]
4+
charset = utf-8
5+
end_of_line = lf
6+
insert_final_newline = true
7+
trim_trailing_whitespace = true
8+
indent_size = 4
9+
indent_style = space
10+
11+
[*.{md,yml,yaml,html,css,scss,js,cff}]
12+
indent_size = 2
13+
14+
# These files are edited and tested upstream in nf-core/modules
15+
[/modules/nf-core/**]
16+
charset = unset
17+
end_of_line = unset
18+
insert_final_newline = unset
19+
trim_trailing_whitespace = unset
20+
indent_style = unset
21+
indent_size = unset
22+
23+
[/assets/email*]
24+
indent_size = unset
25+
26+
[/assets/blacklists/GRCh37-blacklist.bed]
27+
trim_trailing_whitespace = unset

.gitattributes

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
11
*.config linguist-language=nextflow
2+
modules/nf-core/** linguist-generated
3+
subworkflows/nf-core/** linguist-generated

.github/.dockstore.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Dockstore config version, not pipeline version
2+
version: 1.2
3+
workflows:
4+
- subclass: nfl
5+
primaryDescriptorPath: /nextflow.config
6+
publish: True

.github/CONTRIBUTING.md

Lines changed: 57 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,11 @@ Contributions to the code are even more welcome ;)
1515

1616
If you'd like to write some code for nf-core/chipseq, the standard workflow is as follows:
1717

18-
1. Check that there isn't already an issue about your idea in the [nf-core/chipseq issues](https://github.com/nf-core/chipseq/issues) to avoid duplicating work
19-
* If there isn't one already, please create one so that others know you're working on this
18+
1. Check that there isn't already an issue about your idea in the [nf-core/chipseq issues](https://github.com/nf-core/chipseq/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
2019
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/chipseq repository](https://github.com/nf-core/chipseq) to your GitHub account
21-
3. Make the necessary changes / additions within your forked repository
22-
4. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
20+
3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
21+
4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
22+
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged
2323

2424
If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
2525

@@ -30,14 +30,14 @@ Typically, pull-requests are only fully reviewed when these tests are passing, t
3030

3131
There are typically two types of tests that run:
3232

33-
### Lint Tests
33+
### Lint tests
3434

3535
`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
3636
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
3737

3838
If any failures or warnings are encountered, please follow the listed URL for more documentation.
3939

40-
### Pipeline Tests
40+
### Pipeline tests
4141

4242
Each `nf-core` pipeline should be set up with a minimal set of test-data.
4343
`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
@@ -46,12 +46,58 @@ These tests are run both with the latest available version of `Nextflow` and als
4646

4747
## Patch
4848

49-
: warning: Only in the unlikely and regretful event of a release happening with a bug.
49+
:warning: Only in the unlikely and regretful event of a release happening with a bug.
5050

51-
* On your own fork, make a new branch `patch` based on `upstream/master`.
52-
* Fix the bug, and bump version (X.Y.Z+1).
53-
* A PR should be made on `master` from patch to directly this particular bug.
51+
- On your own fork, make a new branch `patch` based on `upstream/master`.
52+
- Fix the bug, and bump version (X.Y.Z+1).
53+
- A PR should be made on `master` from patch to directly this particular bug.
5454

5555
## Getting help
5656

57-
For further information/help, please consult the [nf-core/chipseq documentation](https://nf-co.re/nf-core/chipseq/docs) and don't hesitate to get in touch on the nf-core Slack [#chipseq](https://nfcore.slack.com/channels/chipseq) channel ([join our Slack here](https://nf-co.re/join/slack)).
57+
For further information/help, please consult the [nf-core/chipseq documentation](https://nf-co.re/chipseq/usage) and don't hesitate to get in touch on the nf-core Slack [#chipseq](https://nfcore.slack.com/channels/chipseq) channel ([join our Slack here](https://nf-co.re/join/slack)).
58+
59+
## Pipeline contribution conventions
60+
61+
To make the nf-core/chipseq code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
62+
63+
### Adding a new step
64+
65+
If you wish to contribute a new step, please use the following coding standards:
66+
67+
1. Define the corresponding input channel into your new process from the expected previous process channel
68+
2. Write the process block (see below).
69+
3. Define the output channel if needed (see below).
70+
4. Add any new parameters to `nextflow.config` with a default (see below).
71+
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
72+
6. Add sanity checks and validation for all relevant parameters.
73+
7. Perform local tests to validate that the new code works as expected.
74+
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
75+
9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
76+
10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
77+
78+
### Default values
79+
80+
Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
81+
82+
Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
83+
84+
### Default processes resource requirements
85+
86+
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
87+
88+
The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
89+
90+
### Naming schemes
91+
92+
Please use the following naming schemes, to make it easy to understand what is going where.
93+
94+
- initial process channel: `ch_output_from_<process>`
95+
- intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
96+
97+
### Nextflow version bumping
98+
99+
If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`
100+
101+
### Images and figures
102+
103+
For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 0 additions & 42 deletions
This file was deleted.

.github/ISSUE_TEMPLATE/bug_report.yml

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: Bug report
2+
description: Report something that is broken or incorrect
3+
labels: bug
4+
body:
5+
- type: markdown
6+
attributes:
7+
value: |
8+
Before you post this issue, please check the documentation:
9+
10+
- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
11+
- [nf-core/chipseq pipeline documentation](https://nf-co.re/chipseq/usage)
12+
13+
- type: textarea
14+
id: description
15+
attributes:
16+
label: Description of the bug
17+
description: A clear and concise description of what the bug is.
18+
validations:
19+
required: true
20+
21+
- type: textarea
22+
id: command_used
23+
attributes:
24+
label: Command used and terminal output
25+
description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
26+
render: console
27+
placeholder: |
28+
$ nextflow run ...
29+
30+
Some output where something broke
31+
32+
- type: textarea
33+
id: files
34+
attributes:
35+
label: Relevant files
36+
description: |
37+
Please drag and drop the relevant files here. Create a `.zip` archive if the extension is not allowed.
38+
Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.
39+
40+
- type: textarea
41+
id: system
42+
attributes:
43+
label: System information
44+
description: |
45+
* Nextflow version _(eg. 21.10.3)_
46+
* Hardware _(eg. HPC, Desktop, Cloud)_
47+
* Executor _(eg. slurm, local, awsbatch)_
48+
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
49+
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
50+
* Version of nf-core/chipseq _(eg. 1.1, 1.5, 1.8.2)_

.github/ISSUE_TEMPLATE/config.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
contact_links:
2+
- name: Join nf-core
3+
url: https://nf-co.re/join
4+
about: Please join the nf-core community here
5+
- name: "Slack #chipseq channel"
6+
url: https://nfcore.slack.com/channels/chipseq
7+
about: Discussion about the nf-core/chipseq pipeline

.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 0 additions & 24 deletions
This file was deleted.
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
name: Feature request
2+
description: Suggest an idea for the nf-core/chipseq pipeline
3+
labels: enhancement
4+
body:
5+
- type: textarea
6+
id: description
7+
attributes:
8+
label: Description of feature
9+
description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
10+
validations:
11+
required: true

.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,24 @@
1+
<!--
12
# nf-core/chipseq pull request
23
34
Many thanks for contributing to nf-core/chipseq!
45
56
Please fill in the appropriate checklist below (delete whatever is not relevant).
67
These are the most common things requested on pull requests (PRs).
78
9+
Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
10+
11+
Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/chipseq/tree/master/.github/CONTRIBUTING.md)
12+
-->
13+
814
## PR checklist
915

10-
- [ ] This comment contains a description of changes (with reason)
16+
- [ ] This comment contains a description of changes (with reason).
1117
- [ ] If you've fixed a bug or added code that should be tested, add tests!
12-
- [ ] If necessary, also make a PR on the [nf-core/chipseq branch on the nf-core/test-datasets repo](https://github.com/nf-core/test-datasets/pull/new/nf-core/chipseq)
13-
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
14-
- [ ] Make sure your code lints (`nf-core lint .`).
15-
- [ ] Documentation in `docs` is updated
16-
- [ ] `CHANGELOG.md` is updated
17-
- [ ] `README.md` is updated
18-
19-
**Learn more about contributing:** [CONTRIBUTING.md](https://github.com/nf-core/chipseq/tree/master/.github/CONTRIBUTING.md)
18+
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/chipseq/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/chipseq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
19+
- [ ] Make sure your code lints (`nf-core lint`).
20+
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
21+
- [ ] Usage Documentation in `docs/usage.md` is updated.
22+
- [ ] Output Documentation in `docs/output.md` is updated.
23+
- [ ] `CHANGELOG.md` is updated.
24+
- [ ] `README.md` is updated (including new tool citations and authors/contributors).

.github/markdownlint.yml

Lines changed: 0 additions & 9 deletions
This file was deleted.

.github/workflows/awsfulltest.yml

Lines changed: 31 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,31 @@
1-
name: nf-core AWS full size tests
2-
# This workflow is triggered on releases.
3-
# It runs the -profile 'test_full' on AWS batch
4-
5-
on:
6-
release:
7-
types: [published]
8-
9-
jobs:
10-
run-awstest:
11-
if: github.repository == 'nf-core/chipseq'
12-
name: Run AWS test
13-
runs-on: ubuntu-latest
14-
steps:
15-
- name: Setup Miniconda
16-
uses: goanpeca/setup-miniconda@v1.0.2
17-
with:
18-
auto-update-conda: true
19-
python-version: 3.7
20-
- name: Install awscli
21-
run: conda install -c conda-forge awscli
22-
- name: Start AWS batch job
23-
env:
24-
AWS_ACCESS_KEY_ID: ${{secrets.AWS_ACCESS_KEY_ID}}
25-
AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_SECRET_ACCESS_KEY}}
26-
TOWER_ACCESS_TOKEN: ${{secrets.AWS_TOWER_TOKEN}}
27-
#AWS_JOB_DEFINITION: ${{secrets.AWS_JOB_DEFINITION}}
28-
AWS_JOB_QUEUE: ${{secrets.AWS_JOB_QUEUE}}
29-
AWS_S3_BUCKET: ${{secrets.AWS_S3_BUCKET}}
30-
run: | # Submits job to AWS batch using a 'nextflow-4GiB' job definition. Setting JVM options to "-XX:+UseG1GC" for more efficient garbage collection when staging remote files.
31-
aws batch submit-job \
32-
--region eu-west-1 \
33-
--job-name nf-core-chipseq \
34-
--job-queue $AWS_JOB_QUEUE \
35-
--job-definition nextflow-4GiB \
36-
--container-overrides '{"command": ["nf-core/chipseq", "-r '"${GITHUB_SHA}"' -profile test_full --outdir s3://'"${AWS_S3_BUCKET}"'/chipseq/results-'"${GITHUB_SHA}"' -w s3://'"${AWS_S3_BUCKET}"'/chipseq/work-'"${GITHUB_SHA}"' -with-tower"], "environment": [{"name": "TOWER_ACCESS_TOKEN", "value": "'"$TOWER_ACCESS_TOKEN"'"}, {"name": "NXF_OPTS", "value": "-XX:+UseG1GC"}]}'
1+
name: nf-core AWS full size tests
2+
# This workflow is triggered on published releases.
3+
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
4+
# It runs the -profile 'test_full' on AWS batch
5+
6+
on:
7+
release:
8+
types: [published]
9+
workflow_dispatch:
10+
jobs:
11+
run-tower:
12+
name: Run AWS full tests
13+
if: github.repository == 'nf-core/chipseq'
14+
runs-on: ubuntu-latest
15+
strategy:
16+
matrix:
17+
aligner: ["bwa", "bowtie2", "chromap", "star"]
18+
steps:
19+
- name: Launch workflow via tower
20+
uses: nf-core/tower-action@v3
21+
with:
22+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
23+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
24+
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
25+
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/chipseq/work-${{ github.sha }}
26+
parameters: |
27+
{
28+
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/chipseq/results-${{ github.sha }}"
29+
"aligner": "${{ matrix.aligner }}"
30+
}
31+
profiles: test_full,aws_tower

0 commit comments

Comments
 (0)