.hg repo into a .git repo

[ChristopheBordieu]

Hi team,

Our OpenGrok instances index only Git repos.
Unfortunately, for historical reason, some of these Git repos contain into subpath a Mercurial repo (.hg). Eg:

$OPENGROK_SRC/PROJECT/repo
  .git
  tete/...
  toto/titi/.hg
  toto/titi/tata
  tutu

(I know it is not clean.)

When indexing (project-based), I set depth to 2 in order to look only to path like PROJECT/repo.
No hg command is available on our env. So it is not set into configuration file.

Surprisingly, I have messages into the log indicating the Mercurial repos are discovered but obviously not "working" since no hg command available so that these Mercurial repo paths (whose depth is >2) are invalidated.

WARNING: MercurialRepository not working (missing binaries?): /srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup
WARNING: Failed to get parent for /srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup: java.io.IOException
WARNING: Failed to get branch for /srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup: java.io.IOException
WARNING: Failed to determineCurrentVersion for /srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup: java.io.IOException
SEVERE: Failed to read from process: hg
   java.io.IOException: Cannot run program "hg" (in directory "/srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup"): error=2, No such file or directory
WARNING: Non-zero exit status -1 from command [hg tags] in directory /srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup
WARNING: Failed to get tags for: "/srv/opengrok-src/PROJECT/repoGitWithHgInside/backup/1_backup" Exit code: -1

Is it expected?

OG 1.7.35
INFO: Indexer options: [-m, 256, -v, -c, /srv/universal-ctags/bin/ctags, -s, /srv/opengrok-src, --depth, 2, -d, /srv/opengrok-var, -i, *.swp, -i, *.bin, -i, *.gz, -i, *.rpt, -i, *.tar, -i, *.zip, -i, *.class, -i, *.bz2, -i, *.7z, -i, *.tgz, -i, *.exe, -i, *.jar, -i, *.war, -i, *.a, -i, *.so, -i, *.xlsx, -i, *.docx, -i, *.pptx, -i, *.doc, -i, *.ppt, -i, *.xls, -i, *.png, -i, *.PNG, -i, *.gif, -i, *.GIF, -i, *.bmp, -i, *.BMP, -i, *.jpg, -i, *.JPG, -i, *.jpeg, -i, *.JPEG, -H, -P, -S, -G, -R, /srv/opengrok-var/etc/configuration.xml, -W, /srv/opengrok-var/etc/configuration.xml, -U, http://localhost:8080]
(If -i argument could be case insensitive, it will be cool ;-P)

[vladak]

It's a bug. After the repository scan was parallelized in 8df660a, instead of just the source root directory, all its sub-directories are passed for scanning:

opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/index/Indexer.java

Lines 371 to 376 in c53c7ab

	searchPaths = searchPaths.stream().
	map(t -> Paths.get(env.getSourceRootPath(), t).toString()).
	collect(Collectors.toSet());
	}
	getInstance().prepareIndexer(env, searchPaths, addProjects,
	runIndex, subFiles, new ArrayList<>(repositories));

This means that the depth computations are off by one. Instead of adjusting the default scanning depth value, the computation needs to take this into account.

As a workaround run with --depth 0.

[ChristopheBordieu]

The workaround makes for example the line:
19:37:54 WARNING: MercurialRepository not working (missing binaries?): /srv/opengrok-src/PROJECT/repo/toto/titi
no more displayed.
But there are still warnings related to this Mercurial repo (7 warnings).

[vladak]

Make sure to clean any residue (in the form of RepositoryInfo) from the read-only configuration.

[ChristopheBordieu]

Yes. Forgot this.
New config does not contain the Mercurial repo so next indexing should be clean. Will see starting from tomorrow so on Monday.

[vladak]

Also, not sure if having the same argument for -R and -W makes sense.

[ChristopheBordieu]

Also, not sure if having the same argument for -R and -W makes sense.

Why?
The -R aim is mainly to provide configurations not related to repos, isn't it? It is not an issue if all repo configs are part of it.
So, at the end, once indexing is done, the written one can be the read one.
I don't see an issue managing the -R and -W conf files this way. Tell me if I miss something.

Regarding the workaround, I confirm it works correctly: no more occurrence of warns or errors related to Mercurial. Conf is clean now. Restart of Tomcat does not generate any warnings or errors too: catalina.out is clean now.