Determining a consistent way for developers to manage Addons

OF development
#1

Hey all,

I’m hoping to start a conversation that will turn into a consistent way to develop addons for openframeworks.

When we reach something we think is the best approach, we can create an official document and perhaps make a sticky post on the ‘extend’ forum describing it, as well as a post on the of wiki.

Say we’re making ofxMyAddon that requires ofxDependentAddon that someone else made

Folder structure:
the top level folder should be the same name as the addon
_ ofxMyAddon/_

beneath it there should be at least two sub folders
_ ofxMyAddon/src
ofxMyAddon/example_

the src folder should contain your addon source
_ ofxMyAddon/src/ofxMyAddon.h
ofxMyAddon/src/ofxMyAddon.cpp
…_
where the example should contain the example source and project files (the same style that you would normally find in apps/examples/someExample
_ ofxMyAddon/example/MyAddonExample.xcodeproj
…
ofxMyAddon/example/src/main.cpp
ofxMyAddon/example/src/testApp.h
ofxMyAddon/example/src/testApp.cpp
…
ofxMyAddon/example/bin/data_

if there are multiple examples you can have more than one example folder too

This is advantageous because users who download your addon folder can simply drop it into openFrameworks/addons/ and run the examples since they are at the same folder level as the project files in the apps examples.

Managing dependencies
Here’s where I think there will be some potential controversy. If ofxMyAddon requires someone else’s ofxDependentAddon how do we handle this?
My suggestion is that wherever you store ofxMyAddon you also store next to it a copy of ofxDependentAddon that your addon compiles against. This way if the developer of ofxDependentAddon changes it, it wont’ break yours.

If you’re using github for example, you should fork their addon repository into your account.

Also addons should follow the same github branching pattern that openFrameworks has adopted. Namely that there should always be a branch that compiles against the main releases, and a separate branches to keep up with development oF branch and try new features.

Central place for addons
We should find a way to ensure addons are updated here: http://openframeworks.info/ and in general try to build against the latest addons here if you have dependencies.

I know this is different than how Memo does MSA libs (https://github.com/memo/msalibs) or how I was managing ofxFlightphase addons - but I think one repository per addon will be more sustainable in the long run to have.

What does everyone think?

#2

Yes I agree with everything, with one exception: addon dependencies.

I think it’s not good that everyone copies all dependent addons. This will lead to loads of duplicated addons in repositories all around, and probably loads of confusions. Also, what happens if you got a diamond-dependency, i.e. you use two addons which depend on different versions of one addon?

Alternative suggestion: Develop a mechanism for (mandatory) version numbering in addons (maybe with #defines), and addons are only available in one repository (and this repos network of course). This way, the addon repos stay clean and to the point (the addon they contain), and version compatibility can be tracked/managed in the addon itself, respectively by version numbers. If everyone uses the new git workflow, even in addons, version numbers are coming with that more or less without additional effort, so it’s easy in that regard.

#3

I agree having multiple copies is bad, but worse is if the addon you depend on somehow disappears since you weren’t hosting it. I don’t see any other way around it to be safe really, and is why the forking paradigm of github is nice.

The alternative is centralizing the hosting on openframeworks.info or on a somewhat official openframeworks github.

I don’t think requiring addon developers to do mandatory versions will be very sustainable since it’s a pretty fluid process… but i’m open to suggestions if I could see how that would be done more specifically.

the diamond dependency problem further suggests some sort of centralized hosting system that can become the more or less defacto place to house addons that others should build against.

#4

Thanks for starting this discussion.

The only thing I would say is that addons should not be put on http://openframeworks.info/ since that page is not directly affiliated with OF (if I recall correctly). The addons should really be listed somewhere within the openframeworks.cc domain. The location used to be http://addons.openframeworks.cc and is a logical place. There also used to be a integrated redmine install that allowed people to manage their own, but still integrated into openframeworks.cc. I’m not sure what ever happened to that.

It would be great to have the ability to upload addons and moderators can approve/deny them (to keep spam away). This would allow people to upload without needing core OF developers to update the page manually.

#5

Folder Structure - I’m down - love it when I see this format

Managing dependencies I personally put every addon into every project. I have done this somewhat in shame as I know I am contributing to multiple versions floating around but the truth is that I have only regretted when I didn’t do it.

Central place for addons
What if there were an “official” github repo where a ofxAddon Developer could register their ofxMyAddon? I am thinking like HomeBrew’s formula page. Maybe the commit here would just be a README file with a description of what it does points and points to the dev’s addon repo.

https://github.com/mxcl/homebrew/tree/master/Library/Formula

I agree with the one repo per addon.

#6

I always liked the idea of addons.openframeworks.cc, being a place for people to learn more about the addons + ecosystem of OF, sort of like the libraries of Processing or Arduino. It doesn’t have to list all of them, just some that are “approved” or “recommended”. It also helps a lot when trying to understand what OF is built for and why someone might want to use it.

#7

Cool -

Well it seems like we are in agreement about the file formatting.

So the next step is to figure out where / how we manage these addons?

Do we need an Add-on champion in addition to the core OF champion roles we defined in the other post?

I think in the meantime before their is a centralized place we try to host local, compatible, add on copies so we know they wlll work

#8

I agree with the folder structure thoughts you laid out. I also think we do need an addon wrangler to do stuff like confirm what version something last worked with (maybe putting that in the addons site), pick which ones are recommended, examples, credits.

#9

I think we might want two phases of addons. I think finding an addon wranger is going to be hard especially since we don’t already have a central list of all addons. I think the easiest way to maintain a list would to be allow the addon developer to maintain it. This is originally how it worked when we were using redmine a couple years ago. The problem is redmine is kinda ugly and I think addons were still hard to find. A nicer UI would help.

For example, if there was a form they could fill out that propigated the addon list, then it could have fields for:

Addon Name:
Addon Version:
Last Updated Date:
Last OF Version Tested With:
Link to Addon: (external or hosted on OF)
etc.

People could potentially ‘vote’ on the last compatible version of OF kinda like how wordpress users can say that a module is working with X version of wordpress.

Having a system like this wouldn’t put the burden on a few individuals to update, test, and maintain a list. We could still do that - but I think that could be in conjunction with a system like above. So, we have a list and then we have ‘officially tested/recommended’ list above it or on another page.

I’m not sure what ever happened with the main openframeworks.cc wordpress port. Potentially, the above could be built into wordpress itself. We could create a ‘addon’ content type that has all the fields listed above and allows people to upload their addon. This of course requires someone to actually work on the OF wordpress site so people can start joining it again. I could potentially help with that if it’s wanted, but time (for me) is a bit limited right now.

#10

i push for the following concept:

New addon.xml file which defines for each add-on:

name of addon
author of addon
url or author/addon
addon version number
git uri of addon / tag of addon
N x git uri of addon / tag of dependancy
compatible oF versions

then all these xmls are uploaded to a central server
and little app can enumerate available addons and download the addon (+required dependencies if the user wishes)
each time you tag a new version of your addon, upload a new xml with the appropriate git uri / tag

user browses addons for 007 with little app, is presented with all the latest versions of addons which are compatible with 007

little app pulls addon,

user can give centralised feedback through little app (tips / feedback / quick bugs / quick comments)

little app can be online+offline using existing oF login schema

super optimistic. but entirely worth it imo

#11

generally what i’m pushing for is for pressure on meta-data / consistency / centralisation / usability
and less focus on automation / verbose file listing

(tbh i rarely if ever use the existing xml files)

#12

elliot - love the idea. I wonder how it would work with multiple platforms where libraries may be needed/unavailable

#13

generally with an addon, it’s developed by 1 dev on 1 platform
i think it’s quite rare / unrealistic for many useful addons to come with all the libs for all platforms

part of the metadata could then be (as with previous xml) - supported platforms
linux, win, osx, iOS, Android

I think getting ahold of the libraries if they’re not supplied, is where a centralised comments system would come into its own. Comment thread like:

“Nice addon, had to get libblah.a from http://www.blah.com to make this work on OSX. Is really sweet!”

"I got this running on linux by including the libblah source code at http://www.blah.com/source"

#14

All these:

name of addon
author of addon
url or author/addon
addon version number
git uri of addon / tag of addon
N x git uri of addon / tag of dependancy
compatible oF versions

are great. They could be in a separate XML file, or I suppose just in the .h (people don’t seem to use the addon.xml too often for some reason) but an .xml would be better.

generally with an addon, it’s developed by 1 dev on 1 platform
i think it’s quite rare / unrealistic for many useful addons to come with all the libs for all platforms

I would say that I think that’s the goal though, and one of the real appeals of OF. I hate seeing functionality scattered across multiple addons with different APIs b/c the original developer only cared about OSX/* (I have been guilty of this myself, so it’s as much admission as accusation). One of the real strengths of having a listing of addons would be to see that devA made something, devB forked it and added Win support, and devC did some other clever stuff with it. It might also make a nice enticement to do more cross-platform support.

#15

+1 to what joshua said. xml would be better, cause it’s easier to parse/work with than some stuff in a .h file, in case there’s a need for automation in the future (warning-level of-log entries about wrong versions or whatever)

#16

I think the idea is great, but why not have a web submission form instead of uploading an xml?

I agree with Elliot’s pie-in-the-sky scenario of having a centralized database. but I feel like we need a community-powered loose solution that reflects the way ofx addons are made that doesn’t put the burden on anyone.

I’m thinking something more like a revamp of the infamous pirate pad list:
http://piratepad.net/1TwSWduAae

Where we impose structured categories and a required format list your addon there.

#17

here’s one more idea. this ones a little crazy.

what if we just required people to have their code on github, and use the ofx prefix?

everyone does this already, actually.

then we could write a script that uses the github search api http://develop.github.com/p/repo.html

and do a quick search for anything OF related: http://github.com/api/v2/json/repos/search/open+frameworks

then we have a big list of all the relevant repos. we also have:

  • author
  • homepage
  • download link
  • last update (which gives a hint regarding the relevant version of OF to use)
  • url for repo
  • (potentially) a readme file

maybe the only manual input would be an admin interface that allows you to tick a box yes/no as to whether the addon should be shown on the OF site.

this requires a little more hacking, but i like the idea of something that is just a smart tool that takes advantage of how people already work – rather then telling people they have to work a certain way.

#18

I’m with obviousjim that a web submission form would be good. I think making people be on github is ‘ok’, but that’ll inevitably leave people out.

Since openframeworks.cc already runs wordpress and had a beta period where people could sign up a year or more ago, why not really expand the community? It’s fairly easy to to create a ‘addons’ content type that could have fields for platform, OF version, download link, homepage, repo, whatever we want, etc. That would also allow for commenting and discussion on each addon. Then a page could be made that just lists all of them and they could probably be filtered by field information. This would ultimately expand openframeworks into a unified community. Since things would be integrated into wordpress, we’d be able to showcase them however we’d like. The better OF is showcased to people not familiar with it, the more contributors we’d ultimately have. There’s so many things that could be done (beyond just showing addons), by having this information and more people in the database.

I guess the bigger question is, what is the future for oF? Does it aim to expand it’s community organizationally or develop organically like it has been. If we’re looking for a quick solution, I think the xml, prior suggestions, or simple google form could suffice for propagating an addon document. However, if the goal is to ultimately expand the community and showcase work, then I think a community website driven solution would be better. I think that was originally the plan with the wordpress port, but somewhere along the line it stalled.

[Kyle's list] 1 - oF goals description
#19

neat idea kyle!
seems to be a limit in that search, perhaps extended by a flag, or with github api access.

if we went that way, we could choose the unique term that we search for to be the URL of the extension manager. e.g.
"This extension is available from http://addons.openframeworks.cc/"
then we search on that term

i guess if we went that way, we’d standardise on tags, e.g. “0.2/007/win,osx,linux” means that it’s version 0.2 of the extension, is built to run with 007, and is compatible with win,osx,linux

And then we’d presume all comments / feedback / discussion would be on github for that extension

going back to the other way
i was thinking of a cydia type approach (similar to apt), not necessarily central.
you have a ‘repository’ which is a list of addons, with metadata and download info for each
(but instead of hosting the downloads, have git uri’s instead)
then your standalone desktop app ‘extension app store’ has a list of repo urls, and pulls list of available addons from those repos.
main repo is http://extensionsrepo.openframeworks.cc

Each repo has a web app with the submission form described by jim
We pull the readme.txt as the extension description

Perhaps also enforce a few things at submission time:
* folder structure (some super simple rules)
* an image (multiple images allowed)
* an example

generally i presume almost everyone would use the openframeworks.cc repo
but people would be free to run their own if they like by downloading the repo scripts

on another point
i’m +1 for making git mandatory for addons,
but initially uneasy (+0) about 100% github.

#20

I think having a desktop app makes the barrier to entry much higher. Personally, I don’t want to download an app to be able to get to addons.

I think that addons serve two purposes in terms of community. They 1) expand the code base for existing oF developers and 2) they showcase to everyone what can be done with oF - expanding the overall community. If you have a standalone desktop ‘app store’ or you put everything on github only, you eliminate #2. Plus, i think it’s a bigger task to write a desktop app then it is to use the existing wordpress installation that’s already setup for this - it’s a bit of reinventing the wheel. Cydia works because it’s not officially part of Apple. However, for addons - it should really be officially part of oF IMO.

OpenProcessing, Wordpress, drupal, redmine, chrome, firefox, etc are examples of software communities that all have addons and they all utilize web-based submission and showcasing. I think there’s a lot of examples out there to pull from rather than starting from scratch.

Just my 2 cents :slight_smile: