[Kyle's list] 6 - git workflow

_[as someone suggested this, I will open a thread for the main points Kyle brought up in this 007 megamail. First post is his original point. I could use some help with people adding the main points of what has been discussed since then.]
_

we really need to switch to the branching pattern cristoph brought up
in his email. i think it will be hugely conducive towards a more
regular release cycle, and keeping a good divide between people who
really want to develop the core and people who want to use the
releases. i think we should also drop all the defunct branches from
the repo – i’m pretty sure they’re just confusing people

we also seriously need to go back through the git history and purge
all the extra binaries. this isn’t easy, but it’s possible. i’ve had
conversations with a few people about this, and i think it’s important
to make the repo more like 200 mb rather than the current ~900 mb. if
we don’t figure this out now, it’s just going to keep getting worse.
the usual way people deal with this is by building libraries locally,
dynamically linking things, etc. OF has a different model, but we need
to think about how that can work best with git.

joshua on the “download this, do this” packaging strategy:
Just a quick thought here: I think we’re moving towards a “download
this, do this, then run the examples” mentality that might enable some
of the binaries to be built post-download and would shrink the repo
size considerably. I realize that’s different than “download this,
then run the examples” and I know that’s significant, but there’s some
real advantages to having people build locally as well with
performance optimizations, etc. If this can be done with click-to-run
apps/exe rather than command-line it would keep the process of getting
started easy while also allowing optimizations, advanced config
options, etc.

I think there is a need for discuss this further.
Can anyone tell me whats the advantage of having a seperate development branch (despite only because the article says that) ?
It’s a common practise to develop on the master branch and use this as “HEAD” or “trunk”.
So if you say use recent HEAD, then you want to say use the master branch.
I don’t see any reason for not developing in master branch, but some points against it:
sending a pull request goes automatically to master branch. we will have to treat many “wrong” pull requests.
Master branch is the default branch when forking/cloning, everyone needs to know to switch to dev branch to get the recent trunk/HEAD.
I’m for skipping dev branch and push everything new to master, if there is no good reason to use a seperate branch. Stable versions are marked with tags, thats what they are for.
This is how we worked in the past: use the version from the website or use the git repo.

Simple: say, people go to the git homepage. The first thing they see and the thing they automatically get, is always the current stable version. I don’t think we can expect from everybody to search for the last/current tag and select that, but we can expect everybody to hit a download button.

I think first and foremost, we want people to get the stable version. Otherwise there will maybe be the impression, that oF is full of bugs. If people are willing to accept possible bugs/breakage/incompatibilities to get some cutting-edge, they are normally proficient enough to select/check out the develop branch.

also, the last half year was a mess version-wise, everybody was talking about how they use 007 when it wasn’t released yet, it was almost never clear which commit they were talking about (especially when you stumble upon a discussion a few days later). Using the new method, most people will get/have 007, and we can rely on this information to find/kill bugs.

another reason: if you work on developing core oF, but in the meantime need a know stable version for a project or whatnot, you commit/stash your work in your feature branch, and checkout master for work on your project. yes i know this can also be accomplished with tags, but i think it’s better to have a bit more separation.

about the wrong pull requests: yes, that’s a possible scenario, but only time will tell how often this will occur. The new wiki page and sticky topic in here should help.

if I wanna use the latest stable version, I can download it from the homepage, this is what the downloads are for. No user who is not interested in developing OF core will download from the homepage and should be pointed to that. Everyone else should work with github unstable trunk. This is how we were working in the past. This means, all this redudant and more complicated stuff for users is only why we are too lazy. For me these are only rules which makes it more complicated for new users to get in and therefor should be changed to the minimal complexity.

Don’t you mean “every user” in your second sentence?? :confused:

Using “that’s how we always did it” as an argument is not really valid I think. If things need changing, change them. If not, don’t.
I don’t feel that this workflow is either redundant or overly complex, considering that they can be explained in 3 sentences (like in the sticky topic on here).

For new users to get in they only have to know this: “branch from develop, do your work in your own branch, PullRequest to develop”. The more complex aspects of this workflow (release branches, hotfix branches, tags) are handled only by experienced devs (who have write access to the oF repo), and don’t need to concern outside contributors. that’s good compartmentalisation, if you ask me…

finally, maybe we just have to agree to disagree. :slight_smile:

yes, I meant “every user”

right, so the “master banch mode” is fine only the tags were missing, so here they are.

…but have to be read before committing. not better than zero sentences and a common practise nearly every big project uses in its git workflow (rails, linux, jquery, mongo, …). no one has a seperate dev branch were people should pull in because everyone has its own fork for this.

So let it do in this way like people will expect to work, because the only advantage you mentioned was

  1. having a stable branch for dev’s, which should not be the reason for bothering the whole community
  2. people think OF is full of bugs. not when they use the official packages from the website (what they should do)
  3. messy. this was only a fault of missing tags and missing communication, which can not be fixed with a new branch.

One thing that we might be able to do to mitigate the “oF is full of bugs” problem is make the submissions against openframeworks/openframeworks happen less often. Processing.js has a separate publicly facing repo that just contains stable releases. For the unstable stuff you need to look through the bug tracker and find the commits that add the features that you want and pull them yourself.

Their workflow is detailed here:
https://processing-js.lighthouseapp.com/projects/41284/how-to-contribute

The other thing that they do is have a peer review process for the code that promises at least 3 people have looked at it before it ever makes it into the main repo (owner, peer-reviewer, super-reviewer). I think this is something that we could potentially benefit from to keep the number of bugs down.

The other thing that they are doing here is using a far more robust bug tracker than the github one. This may have more to do with the bug hunt http://forum.openframeworks.cc/t/organised-bug-extermination-drive/6898/0 but perhaps a new tracker may help get more organized…
https://processing-js.lighthouseapp.com/projects/41284/how-to-contribute

yes, i also think having a master that is exactly the same as the last version is not very useful. i think bugfixes should be done against master, and merged in develop, while new features should be done against develop

I have a question.

Reading how this works
http://forum.openframeworks.cc/t/git-how-to-contribute-code-to-of/6881/0

If a bug is found in 007 and fixed in github, will this go into openframeworks.cc/download/ into an updated 007, or do all bug fixes go towards the next release?

right now there are 199 issues on github, most from before 007 was released. i feel like the issues should just relate to the lastest 007 release.

also hardly any of the issues have labels. it would be better if everything was labelled with bug / feature / 007 etc as fixing bugs should be first priority, adding features for next release ?

https://github.com/openframeworks/openFrameworks/issues?page=1&state=open

it feels like the master should be 007 with bugs fixed as they come in, and develop should be for the next version?

if it’s fixed in github, it will go into the next release (if it’s a bugfix release, it would have a minor version number, 0071 or something, which would happen more often as detailed in the release schedule thread).
If you would just “update” 007 without incrementing versions, this would lead to confusion because it would be possible that two people have versions with different functionality (i.e. certain bugs are fixed or not), but both would be called 007. this would make addon maintenance (against certain core oF versions) and bughunting really complicated, because you would not know “which” 007 people are talking about.

@bug labelling: yes this would certainly help. that’s a part of the reason for my call for a bug hunt.
labels like 0071 would also enable us to set milestones, i.e. assign certain bugs a target release. if a bug is labeled 0071, it has to be fixed before 0071 can be released (there’s a mechanism for this in the bug tracker). if a more severe bug (e.g. big feature or api change) is labeled 0080, there’s more time to fix/implement it, cause it has another milestone.

it makes more sense for a bug found in 007 to be labelled 007

then the mechanic who comes along later can know ‘aha, that bug was in 007, but now we’re on 009 it’s already been fixed because system X has been completely rewritten and checked.’

Rather than
‘i want this fixed for the next release, so i’ll tag it 0071’

“it’s already been fixed because system X has been completely rewritten and checked.” - then it should just be marked as “fixed”… :wink:
‘i want this fixed for the next release, so i’ll tag it 0071’ is the way it should be… what’s the added value of knowing in which release the bug occured first??

see for example this:
https://launchpad.net/inkscape/+milestone/0.48.2
all bugs are fixed, so packagers know they can start work on packaging up inkscape to release the next version.
and these are all bugs targeted for 0.49, and their status:
https://launchpad.net/inkscape/+milestone/0.49

totally agree! never update without changing the version number.

@bug labeling:
you are both right and wrong:

(given the fact that we use github issue tracker)
labels should never be used for versioning. not for “fix this in 0071” or “its broken in 007”.
if you want to have a bug fixed in for 0071 then it should be get the label “bug” and assigned to the milestone 0071. then you could do a priority: “bugs”, "minor, “feature”, …! …and should only release if every bug is fixed for a certain milestone.
if you have a bug then its a bug from the latest version, no need to label this as “007” or something like that. its a bug and its still in current release, so it needs to be fixed.
so labels and milestones should be used for two different things.

ok, this is my fault apparently - i had the wrong impression that the github issue tracker used labels for milestoning, when in fact there is a separate interface for milestones. sorry 'bout that.

+1 on using milestones instead of ambiguous labels

+1 on arturo’s style – slightly deviating from the ‘successful git branching model’ (SGBM) – but i think it’s most effective for bug fixes to be done against master and pulled into everything else (rather than a separate release branch). the reason for this is that the website has ‘downloads’, which serve more like the ‘master’ branch in SGBM.

regarding how libraries are brought into OF, i like this model:

https://github.com/deanm/plask/blob/master/fetch-deps.sh

it makes it possible for people to download + compile libs themself, or use a collection of libs we provide by running a script.

but its not possible to tie a specific version of OF to a specific version of the libs and may be leads to confusing behavior if libs were updated without any version number?!

this model would only be used for git development. releases would still contain the libraries pre-packaged. this is just a technique for getting the libraries out of git without raising the bar to contributions from developers.

but it would be hard to go back in dev history for bisecting or something like that, because we only have the packages working for recent dev version of OF.
A possible solution for that would be checking the shellscript in git and explizit pin down the lib versions in it.