Closed Bug 883711 Opened 11 years ago Closed 10 years ago

initiative to separate gaia module for reuse

Categories

(Firefox OS Graveyard :: Gaia, defect)

Product:
Firefox OS Graveyard
Component:
Gaia
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: gasolin, Assigned: gasolin)

References

Details

Attachments

(1 file)

script to generate gaia in separate folders
1.36 KB, application/x-sh
Details 
Based on discussion of
* Git submodule proof of concept
https://groups.google.com/forum/?fromgroups#!searchin/mozilla.dev.gaia/git$20submodule$20proof$20of$20concept/mozilla.dev.gaia/3L8MQzpE81E/2QlRK1kKzdkJ
* Repo experiment [was : Git submodule proof of concept]
https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.gaia/CCJUzs9vHk4

We'd suggest to come out a proposal/small script to clearly describe/generate which part of gaia could be organized in different git repositories (both git submodule or repo need that). 

the proposal could be a base that help communities and managers do the wise decision.

I think /shared directory is a good place to kick off.
FYR there's a 3rd-party building blocks repo
https://github.com/buildingfirefoxos/Building-Blocks

We should take 'separate an official git for building block' into concern that even mozilla is value the choice of web, and we'd not preferred to bless any UI style. 

Developer still try to use building block for firefox os native looks and feel when they first expose the firefox os.
My init take will start from 4 directories:

- gaia-exp (gaia remove shared folder)
- gaia-shared (shared folder remove building block)
- gaia-shared-style (stable building block)
- gaia-shared-style-unstable (unstable building block)


Will do it in my github and add another .git to host default.xml for 'repo' command.

though I prefer merge 'style'/'style-unstable' into 'style' (bug 847777), but lets get start with unobtrusive way.
Assignee: nobody → gasolin
I've created https://github.com/gasolin/gaia-repo experimental repo for test purpose.

You can make your own gaia directories by using attached script.

check default.xml that will fetch from 4 git repositories defined in comment 2
https://github.com/gasolin/gaia-repo/blob/master/default.xml
Due to the recent followup discussion (Splitting out gaia into smaller repos (more! sorry sent it early) https://groups.google.com/forum/#!topic/mozilla.dev.gaia/3Ye0uGtW-iw

Chance to splitting gaia reposis is rising. I'd resume the experiment of working with 'repo' and 'git' commands in daily process. Then come out a document or propose necessary actions.
Done the first gaia submodules update from current gaia-master, its simple than I expected:

1. in origin gaia-master folder, checkout gaia-exp branch and rebase it

$ git checkout gaia-exp
$ git rebase upstream/master
$ git push --force origin gaia-exp

results in 
https://github.com/gasolin/gaia/tree/gaia-exp

2. overwrite submobules by copy shared/ from gaia-master folder

$ cp -R gaia/shared/ gaiaexp/shared/

$ cd gaiaexp
$ repo sync

commit all changes and push submodules separately.

results in 
https://github.com/gasolin/gaia-shared


The trick is 'repo' checks out the revision on a detached head, which "git branch" reports as "(no branch)" [Ref]. So we need explicitly add a branch before checkin code or push back to github.

$ git checkout --track -b master gaia/master


Ref: https://groups.google.com/forum/#!topic/repo-discuss/q6gzEE-crgI
have a workable process elaboration in slide http://gasolin.github.io/gaia-repo/
feedbacks are welcome.
Hi Fred,

I was experimenting with repo a bit, and was unable to find a clean way to copy multiple versions of a project.

Basically I think that the solution we use should be able to have each app specify it's own version of shared code. Do you think repo will be useable for this?

I tried listing multiple projects at different revisions, but you get an error, fatal: duplicate project gaia-shared in <...>gaia/.repo/manifests/default.xml


Thanks!
Flags: needinfo?(gasolin)
Since repo can detect and support git submodules, it might be worth checking if using multiple submodules (but repo to make sure people don't shoot their feet with the submodules) would work.
(In reply to Andrew Sutherland (:asuth) from comment #9)
> Since repo can detect and support git submodules, it might be worth checking
> if using multiple submodules (but repo to make sure people don't shoot their
> feet with the submodules) would work.

If that works I think that would be ideal! I'll implement a PoC if I can figure out a clean implementation with repo and submodules.
I took a further look and it appears that in the 'stable' version of repo, submodules are not supported. There is some code to handle them here: https://github.com/android/tools_repo/blob/master/manifest_submodule.py. I found this code buggy, and I couldn't get it working properly.

It might be worthwhile to have a repo expert taking another look, but I don't believe repo will be able to cover our use cases. I'm going to spend some time reviving my submodule approach, and wrapping the commands within the makefile to protect people from shooting themselves.
Recall that the hg mirroring that is performed for buildbot is not capable of handling submodules; cc'ing rel-eng for more details.
Adding in a comment from an earlier email thread on this subject:

> tl;dr: git-submodules will not work; separate repositories accessed via
> the "repo" tool will work for the official builds.
> 
> Longer story: at the moment, our FFOS build and l10n processes both
> require hg repositories. That means that we convert the gaia git repos
> to hg internally. Sub-modules are one feature that does not map at all
> between the git and hg worlds.

Based on comment 11, it sounds like further investigation has shown that the repo tool will also not work. That's unfortunate.

A sub-module based initiative will need to include solutions to integrating with the l10n & release build systems.
Hi kevin,

I think track multiple versions of a project with 'repo' is pretty simple (just via git branch).


You can see how android manage their versions
https://github.com/android/platform_manifest/blob/android-4.2.2_r1/default.xml

1. set all revision to android-4.2.2_r1(git branch name) by set <default revision="refs/tags/android-4.2.2_r1">
2. add 'revision' attribute separately on each <project> tag


You can create any new branch to change default.xml, user just use

$ repo init [path].git -b [branch-name]

to get specific version set by repo command.



If I did not explained clearly, please ni? me again
Flags: needinfo?(gasolin)
1. set all revision/2. add 'revision' attribute separately

are two different ways to track multiple versions of a project. Android use the first one.
kevin,

I've come up with a github project with instruction

https://github.com/gaia-local/gaia-local-repo/tree/tablet-master


which can:

1. get gaia-master to the created folder (mozgaia)
2. put the customization project's branch(tablet-master) in the target(mozgaia/distribution) folder.


With that I think its more easy to compare each branch and figure out how multiple versions of a project works with 'repo' command.
Flags: needinfo?(kgrandon)
Ok, I guess I'll just need to see this in action as you envisioned it. I was under the impression that we wanted to be able to have multiple versions of modules inside of gaia at different revisions. E.g., calendar requests building blocks v1, and email uses v2. I'm still not seeing how this is possible - have you tried for example pulling in the same remote with two different tags? I get an error.

The other alternative is to have each application be it's own module as well. If each app lives in it's own repo, gaia is more modular in nature and the duplicated-project issue might solve itself. (Note I have not tested this)
Flags: needinfo?(kgrandon)
with 'repo', pulling in the same remote with two different tags will get an fetal error though.
Have revised my origin take.


Now its possible to use repo with gaia directly by following the instructions
https://github.com/gasolin/gaia-repo


Basically `repo sync` is able to take care `git clone` and save typing `git rebase` for daily work.

The `repo start` could used to create branch `git checkout -b <branch name> upstream/master`

And it does not change the current workflow. The flexibility is we could use `repo sync <project>` to only update a subproject if we did have sub projects.
I should specify that 972481 is a bit of a "tentative" dependency since it's only necessary if we use git submodules.
Moving question on 'repo' usage here

(In reply to Kevin Grandon :kgrandon from comment #2)
> So I did some investigation of using repo with shared libraries in the past,
> and one thing that did not work was that we would want to be able to include
> different versions of the library within the same project. Some people said
> that this was possible, but I did not have luck when I went to implement it.
> If anyone can prove that it does work I would say repo would work just fine.
> E.g., we would want to be able to pull in different versions as follows:
> 
> gaia/foo/ExternalLib@v1.1/
> gaia/bar/ExternalLib@v2.0/

:gasolin -- do you have a work around for this situation?
Flags: needinfo?(gasolin)
Yes I found its possible with repo, see

https://github.com/gasolin/demo883711


I've a project called fxosbmi that have several branches. with the https://github.com/gasolin/demo883711/blob/master/default.xml file I can sync different branch to different folder.

the instructions are:

$ mkdir demo && cd demo
$ repo init -u http://github.com/gasolin/demo883711
$ repo sync
Flags: needinfo?(gasolin)
Nice! If that's the case I think we should just start using repo. Can we begin with building blocks only?
as comment 19, its possible to use repo with gaia directly by following the instructions
https://github.com/gasolin/gaia-repo

Without separate any module and you can get some syntax sugar from it. ex: 

run `repo sync` instead of 'git merge upstream/master'
run `repo start issue-xxxxxx gaia` instead of 'git checkout -b issue-xxxxxx upstream/master'
The origin goal is fulfilled. To integrate repo usage with gaia, Build script seems the next stop that need to be resolved.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: