mbed-tools deploy is limiting the git fetch specification

[lefebvresam]

Description of defect

When checking out a project with "mbed-tools deploy", the [remote "origin"] section in the config file of the git project checked out is limiting the fetch specification for remote branches.

$ git fetch
$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

cat .git/config [remote "origin"] section shows

[remote "origin"]
url =
fetch = +refs/heads/master:refs/remotes/origin/master

should be:

[remote "origin"]
url =
fetch = +refs/heads/*:refs/remotes/origin/*

If you then do "git fetch" you will be able to see the remote branches with "git branch -a".

$ git fetch
...
$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/r113-fix-git-conversion
  remotes/origin/r151-fix-git-conversion
  remotes/origin/r17-fix-git-conversion
  remotes/origin/r174-fix-git-conversion
  remotes/origin/r239-fix-git-conversion
  remotes/origin/r48-fix-git-conversion
  remotes/origin/r63-fix-git-conversion
  remotes/origin/r65-fix-git-conversion
  remotes/origin/r97-fix-git-conversion

Resolve temporary with:

git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

There is another issue where the CLI2 "mbed-tools deploy". The repo is not checked out fully. I have seen that the history is not complete if you compare with a native GIT clone.

Target(s) affected by this defect ?

The command "mbed-tools deploy"

Toolchain(s) (name and version) displaying this defect ?

Using docker session:

docker run -it --privileged --mount=type=bind,source="$(pwd)",destination=/var/mbed -w /var/mbed -v /dev/disk/by-id:/dev/disk/by-id -v /dev/serial/by-id:/dev/serial/by-id -v /run/udev:/run/udev:ro ghcr.io/armmbed/mbed-os-env:master-latest

What version of Mbed-os are you using (tag or sha) ?

Mbed 1.10.5

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

CLI 2

How is this defect reproduced ?

Clone a GIT project containing remote branches with CLI 2 command "mbed-tools deploy" from source *.lib.

[multiplemonomials]

Hmm, I'm not sure this is really an issue for Mbed CE, because Mbed CE uses submodules the normal way instead of using the mbed-tools command.

However, Mbed CE still does use shallow submodules, so I suppose that something similar to this is still happening. The reason we use them is, it trims ~640MB of git history for the mbed-os submodule down to ~88MB, making each project folder significantly smaller.

If you check here on the wiki, I put the steps to "unshallow" the mbed-os submodule. Following those steps will update it so that you can access all the Mbed branches normally, at the cost of ~500MB additional hard drive space.

[lefebvresam]

I think for a new development it can be desirable to know what branches exist and to browse the history with Sourcetree or whatever else which gives more functionality than just looking into github. This ticket was before my knowledge that the Mbed arm was dead. I agree to make a 'unshallow' topic to make poeple aware of it.

[multiplemonomials]

so do you think that the unshallow page should be moved somewhere else?

[lefebvresam]

It's a good place but I would change the title to make it more clear.

[lefebvresam]

If you did the unshallow procedure, do you have to set shallow back to false in .gitmodules also?

[JohnK1987]

https://github.com/mbed-ce/mbed-os/wiki/New-Project-Setup-Guide#note-converting-the-mbed-os-submodule

[lefebvresam]

Is the unshallow procedure interfering with:

[submodule "mbed-os"]
	path = mbed-os
	url = https://github.com/mbed-ce/mbed-os.git
	shallow = false

in .gitmodules?

[JohnK1987]

When you use just git submodule add --depth 1 https://github.com/mbed-ce/mbed-os.git mbed-os then .gitmodules looks like

[submodule "mbed-os"]
	path = mbed-os
	url = https://github.com/mbed-ce/mbed-os.git

After call git config -f .gitmodules submodule.mbed-os.shallow true

[submodule "mbed-os"]
	path = mbed-os
	url = https://github.com/mbed-ce/mbed-os.git
	shallow = true

Opposite boolean in the command above will cause opposite settings in .gitmodules.
But if you know you want complete repo during creation of new project then you just skip step 4 from the guide. If you did not skip it, then you can later use steps descripted here https://github.com/mbed-ce/mbed-os/wiki/New-Project-Setup-Guide#note-converting-the-mbed-os-submodule. That is how i understand it.

[lefebvresam]

I think if you did the procedure to convert the mbed os submodule that you also have to update the .gitmodules with git config -f .gitmodules submodule.mbed-os.shallow false otherwise when you do a git submodule update this will be reverted.

[JohnK1987]

Ok, i will look on that.

[JohnK1987]

I think if you did the procedure to convert the mbed os submodule that you also have to update the .gitmodules with git config -f .gitmodules submodule.mbed-os.shallow false otherwise when you do a git submodule update this will be reverted.

You right @lefebvresam the shallow state in the .gitmodules remains same after the whole procedure. Anyway the history is not reverted back to the master only when i use git submodule update.
Any ideas?