| lmacken | Hey everyone, I'm Luke Macken and I'm going to talk for a little while about bodhi. | 15:01 |
|---|---|---|
| lmacken | You can grab my slides here: http://lmacken.fedorapeople.org/bodhi.pdf | 15:01 |
| lmacken | I'll start by giving a brief overview of the history of Fedora updates, and then dive into some of the technical and practical aspects surrounding the new update update system for Fedora 7, bodhi. | 15:02 |
| lmacken | Feel free to jump it at any time with questions! | 15:02 |
| lmacken | So.. pushing updates previous to FC4 was extrememly painful. | 15:02 |
| lmacken | Just as it is done today, developers have to build their package by hand, and do whatever testing they deem to be sufficient for their update. | 15:02 |
| lmacken | From here they had to then edit a flat-file in CVS, assign themselves an update ID, and provide basic details such as if it is a testing/final, security related, along with an embargo. | 15:03 |
| lmacken | This file was frequently out-of-date, and was not very much fun to write a parser for :( | 15:03 |
| lmacken | From here, the developer then had to send an email to release engineering, and wait for a response, then generate and send the update notifications by hand to the appropriate mailing list. | 15:03 |
| lmacken | This process was very strenuous not just to package maintainers, but especially to release engineering. The abundance of unnecessary context switches caused this procedure to be extremely painful, and made developers not want to push out updates. | 15:04 |
| lmacken | So, two summers ago began my internship with Red Hat, where I was tasked with creating a web interface to streamline the update process. | 15:04 |
| lmacken | using postgres, mod_python, and nasty chunks of code from various scripts (*cough* multilib-file-of-doom), I created a system that is still being used today to push Fedora Core 6 updates. | 15:05 |
| lmacken | It got the job done, but was far from an optimal solution. First off, it was internal to Red Hat --which only shed light upon the blatant shortcomings of Fedora development. | 15:05 |
| lmacken | Secondly, not only did I know it would never see the light of day, but I was never able to even use my own tool -- so I was fairly content with it sucking :) | 15:05 |
| lmacken | On the other hand, there were definitely some good innovations to come out of this endeavor. | 15:05 |
| lmacken | I worked with Mark Cox, Seth Vidal, and Jeremy Katz on creating a metadata format used to describe package updates, and pretty much implemented the whole stack of software to utilize it. | 15:06 |
| lmacken | THis entails generating the metadata server side, stuffing it into the repos (using modifyrepo, which is now being shipped with createrepo), then parsing it client-side (using the yum.update_md module that seth and I wrote), and displaying the details in pup/puplet. | 15:06 |
| lmacken | Various tools have since come to utilize this extended metadata as well, such as the yum-security plugin, written by James Antill. | 15:07 |
| lmacken | (Note: the lack of updateinfo.xml.gz in Fedora 7 is a legitimate regression. Due to the sudden change in the way bodhi pushes updates, this clump of code has yet to be reworked. It will be there very very soon, I promise :) | 15:07 |
| lmacken | So, enough about legacy software... bodhi has since emerged. | 15:07 |
| lmacken | I'm going to iterate over some of the goals of bodhi, and briefly explain how it has or is in the process of accomplishing them.. | 15:08 |
| lmacken | - Provide an intuitive interface for developers and release engineers to manage pushing package updates for multiple distribution releases. | 15:08 |
| lmacken | To date, bodhi has pushed almost 1,500 updates for Fedora 7, and will eventually handle updates for Enterprise Extras, as well as future releases of Fedora. With very minimal context-switching, the process is fairly trivial, and alleviates much pain from the release engineering side of things. | 15:09 |
| lmacken | - Ensure package quality and repository sustainmentwith automated testing | 15:09 |
| lmacken | Bodhi makes sure each build is properly tagged in Koji, verifies that it does not break any update paths, checks each associated bug for potential security keywords, and performs other various sanity checks. | 15:09 |
| lmacken | The other day I started writing some code to do buildroot verification, and hopefully dependency checking will be right around the corner as well. | 15:09 |
| lmacken | - Encourage community testing and involvement through allowing people to easily test updates and provide feedback | 15:10 |
| lmacken | Bodhi now suppors update 'karma', which is a number that reflects the amount of positive/negative feedback that it has received. | 15:10 |
| lmacken | As you can see in the screenshot on my slides, developers/testers will easily be able to chim in quickly and say that an update works fine for them, or complain very loudly that it is broke and should not be released. | 15:10 |
| lmacken | - Provide a modular framework that will allow future integration to various other QA and developer tools. | 15:11 |
| lmacken | Bodhi is a modular part of the Fedora Infrastructure stack.. It utilizes the Koji buildsystem for tracking RPMS, composes repositories using Mash, and will soon be talking to the Package Database. | 15:11 |
| lmacken | A bit later, I will discuss some ideas for features that will allow bodhi to integration with some pretty sweet pre-existing tools. | 15:12 |
| lmacken | The next slide (#7), shows a diagram of the TurboGears framework. | 15:12 |
| lmacken | Bodhi is modeled almost identical to that diagram, except that we recently switched from MochiKit to JQuery :) | 15:12 |
| lmacken | The beauty of TurboGears lies in it's modularity.. as we can easily sub out various parts of the stack with ease. | 15:12 |
| lmacken | The next polymorphism of bodhi will most likely be the switch from kid->genshi, which will give us a big speed boost, as well as many more useful constructs. | 15:13 |
| lmacken | Aside from some unicode-related explosions, SQLObject has been great to us so far, so I don't see any immediate need to switch to SQLAlchemy. | 15:13 |
| lmacken | So.. about a week or two before Fedora 7 was released, we decided to change the way bodhi "pushes" updates. The "old style" pushing consisted of a single updates-stage in which updates were added/moved/removed by the system itself. | 15:14 |
| |DrJef| | lmacken, can we take advantage of the modular aspect to do some of the verification on community controlled hardware? | 15:14 |
| lmacken | This had its benefits: being fairly low maintenance, and the only real overhead was createrepo -- but it brought upon us more cons than pros. | 15:14 |
| lmacken | |DrJef|: sure, I don't see why not? Although you may need to elaborate on what exactly that entails to be sure. | 15:14 |
| |DrJef| | lmacken, like plugging new qa tests periodically that are proposed for integration into the standard tests | 15:15 |
| lmacken | yeah, definitely feasible.. and hopefully we'll have a nice test lab (beaker) for doing that sort of stuff.. more on that later :) | 15:15 |
| lmacken | So, Bill Nottingham created this spiffy little tool called mash, that essentially takes a koji tag (ie: dist-f7-updates-testing), and composes a fresh repository of RPMS. It handles multilib sanely (if such a thing exists), and does a lot of grunt work for us. | 15:16 |
| lmacken | So I wrote a Masher for bodhi, which is a multi-threaded mash dispatcher. It's a TurboGears extension that runs alongside of bodhi, and is able to rollback any changes if something explodes in the push process. | 15:16 |
| lmacken | It seems to be doing a fairly good job at the moment. | 15:16 |
| lmacken | On slide #9, you'll see a diagram of bodhi's database model. | 15:17 |
| lmacken | These are the core data structures of this system, and are implemented using the SQLObject object relational manager. | 15:17 |
| lmacken | The beauty of the MVC design is that we are able to keep all of the corresponding logic inside of these database-driven objects, making them fairly self-sufficient (ie, each PackageUpdate knows how/when to send it's update announcement, Bugzilla's automatically fetch their title upon creation, etc). | 15:18 |
| lmacken | This gives us a lot of power. | 15:18 |
| lmacken | So now I'll quickly go over the basic usage of the system.. | 15:18 |
| lmacken | From a developers standpoint, bodhi comes into play after you have built your package and are ready to release it to updates{,-testing}. | 15:18 |
| lmacken | Here you fill out a basic web-form, or eventually runa command-line tool, to add the update to the system. | 15:18 |
| lmacken | From here you request that your update be pushed to either the Testing or Stable repository. | 15:18 |
| lmacken | At this point, you can sit back and relax (or go do something productive). You'll get an email when your update has been released, and when people comment on it. | 15:19 |
| lmacken | You can see a few screenshots of bodhi's web interface on slides #11-13. | 15:19 |
| lmacken | So.. from here, if it is a testing update, it needs to be tested. | 15:19 |
| |DrJef| | lmacken, just the owner of the package for now? or are the emails going out to co-maintainers as well? | 15:19 |
| lmacken | |DrJef|: once the package database gains the proper hooks, we'll be able to handle co-maintainers.. at the moment bodhi knows nothing about them. | 15:20 |
| lmacken | Bodhi helps facilitate QA testing by allowing testers to +1/-1 updates, which helps ensure we don't push obviously broken crap to our users. | 15:20 |
| lmacken | I added a 'stable_karma' configuration option the other day that will allow bodhi to automatically mark updates as stable once they reach a given karma (default is 3). | 15:20 |
| lmacken | (this is where a little Bugzilla widget may come in handy for quickly filing bugs against the update component when necessary) | 15:21 |
| * lmacken lets his hands cool down for a sec. | 15:21 | |
| lmacken | So, I've tried my best to make bodhi as easy to hack on as possible. As you can se on the slides, getting a local bodhi instance up and running is trivial. | 15:21 |
| lmacken | A few weeks ago I pickled (serialized) bodhi's production database, and added it to our mercurial repo. So, with a single command you can populate your own bodhi instance with 1,034 real updates. | 15:21 |
| lmacken | I've written a handful of test cases using nose, which is a discovery-based unittest extension. I have found that when implementing a new feature, it's easiest to write a test case for it first, then eventually migrate the stable code into the system itself. | 15:22 |
| lmacken | Nose has definitely been a life-saver... or maybe even a life-style ;) | 15:22 |
| lmacken | So.. what lies ahead for bodhi ? | 15:23 |
| |DrJef| | lmacken, the clientside tool to help updates-testing commenting is interesting, it would need to keep up with which packages on the local system are still need comments in bodhi | 15:23 |
| lmacken | yep | 15:23 |
| lmacken | A command-line tool is right around the corner. | 15:23 |
| lmacken | Toshio and I have been working on creating a generic BaseClient used for writing command-line tools for TurboGears applications. | 15:23 |
| lmacken | It's pretty amazing actually, because it allows us to write tools that utilize the same exact methods that are called when you visit the page with a web browser... just by simply exposing our controllers as JSON. | 15:24 |
| lmacken | fun stuff. | 15:24 |
| |DrJef| | lmacken, are the bodhi comments tied into bugzilla? | 15:24 |
| lmacken | Also, an RSS controller will be rolling out in the near future as well. At this point I also plan on making bodhi accessible for public consumption. | 15:24 |
| lmacken | |DrJef|: nope | 15:24 |
| lmacken | which is why I mentioned having a little widget to interact with bugzilla may be handy :) | 15:24 |
| lmacken | Since bodhi is essentially the central point of a package between the developer and the user, it has much potential to improve the way we sustain our distribution and utilize our community. | 15:25 |
| lmacken | I've come up with some interesting potential ideas for future bodhi features: | 15:25 |
| |DrJef| | lmacken, yeah being able to convert negative karma comments into trackable tickets would be useful | 15:25 |
| lmacken | Upstream monitoring. For those unaware, there exists a tool called FEver that is being used to monitor upstream projects for new releases. Automating this functionality inside of bodhi could potentially kickass. | 15:26 |
| lmacken | |DrJef|: yeah, definitely. | 15:26 |
| lmacken | (also, if i haven't mentioned it yet, all of the projects that I'm talking about are linked on the last slide of my presentation) | 15:26 |
| lmacken | another potential feature... | 15:26 |
| |DrJef| | lmacken, is FEver still using a formatted wikipage? | 15:26 |
| |DrJef| | lmacken, for its configs? | 15:26 |
| lmacken | |DrJef|: I believe so | 15:26 |
| lmacken | i haven't played around with it personally.. I'd rather port it to bodhi first ;P | 15:27 |
| |DrJef| | lmacken, moving that to using the packagedb.....would make sense | 15:27 |
| lmacken | yeah. | 15:27 |
| |DrJef| | lmacken, so maintainers can store the location info in the packgedb... and other people can also make use of it | 15:27 |
| lmacken | Beaker integration! The beaker project is still under development (I hope?), so this feature takes the back burner. But eventually, bodhi will be able to kick off automated regression-tests to an external test-lab. | 15:27 |
| |DrJef| | lmacken, in scripts | 15:27 |
| lmacken | yep | 15:28 |
| lmacken | exactly | 15:28 |
| lmacken | fedora-qa integration. | 15:28 |
| lmacken | Aurelien Bompard wrote an awesome tool that can be used during initial package reviews. The goal of the script is to test all the MUST items in the PackageReviewGuidelines that are not covered by rpmlint. | 15:28 |
| lmacken | Adding these various tests into bodhi would allow us to ensure that our packages maintain compliance with our guidelines throughout their lives. | 15:28 |
| lmacken | So.. the moral of the story is that Fedora has made vast improvements in the way it handles updates, and has the potential to greatly enhance the way we develop and maintain our distribution.. | 15:29 |
| lmacken | That's pretty much all I've got.. thanks for "listening" everyone :) | 15:29 |
| lmacken | Questions/comments ? | 15:29 |
| abadger1999 | |DrJef|: Thinking about reimplementing launchpad? | 15:30 |
| lmacken | :) | 15:30 |
| |DrJef| | abadger1999, gdk and I have talked a lot about exactly that | 15:30 |
| |DrJef| | abadger1999, but this time... open | 15:31 |
| abadger1999 | :-) | 15:31 |
| |DrJef| | lmacken, lets talk qa | 15:31 |
| lmacken | |DrJef|: lets. | 15:31 |
| abadger1999 | |DrJef|: Did you see that canonical open sourced the ORM layer of launchpad? | 15:31 |
| |DrJef| | lmacken, where's the low hanging fruit for organizing volunteer manpower for qa | 15:31 |
| |DrJef| | abadger1999, layer.... | 15:31 |
| lmacken | |DrJef|: well, I think we have alot of people using bodhi already... but mostly in their own little package universe.. I think if we made it more community-based and interactive, people would use it. | 15:32 |
| |DrJef| | lmacken, greg and i got to face to face a bit durring rhh about doing something drastic concerning bugzilla | 15:32 |
| abadger1999 | Exactly. Release the piece that we already have decent alternatives to. Thanks for trying but.... :-) | 15:32 |
| lmacken | so making it trivial/fun to test stuff out, and give feedback. | 15:33 |
| |DrJef| | lmacken, are we going to have to provide a better means for users to downgrade testing packages? | 15:34 |
| lmacken | for example, I'm going to be adding a right-click menu to the update list that will allow people to "Works/Broke" with 2 clicks of a mouse. | 15:34 |
| lmacken | hmm.. most likely. | 15:34 |
| lmacken | that's a good point.. definitely something we need to take into consideration. | 15:35 |
| |DrJef| | lmacken: have you had any discussion with the gui tool builders concerning that | 15:35 |
| lmacken | |DrJef|: nope.. who are the gui tool builders ? | 15:35 |
| |DrJef| | lmacken, jeremy | 15:35 |
| lmacken | No, we haven't discussed it. | 15:36 |
| |DrJef| | lmacken, how far can we take general automated testing before we get into application specific tests? | 15:37 |
| lmacken | |DrJef|: well, I think fedora-qa is a great start, since it is based upon our guidelines.. if we want more tests, we can enhance our packaging policy at the same time. | 15:38 |
| lmacken | but i don't think bodhi should handle package-specific tests.. that's what beaker is for. | 15:38 |
| lmacken | I mean, the code (RHTS) is already written, it's just a matter of deployment. | 15:38 |
| |DrJef| | lmacken, no not bodhi | 15:39 |
| |DrJef| | lmacken, deployment..and teaching people how to write tests for it | 15:39 |
| |DrJef| | lmacken, there's a learning curve | 15:39 |
| lmacken | yeah, most definitely.. I have yet to play with any of RH's internal QA tools.. so I'm fairly ignorant in that matter. | 15:40 |
| lmacken | but will and I have discussed in the past creating a community around writing package-specific test cases. | 15:40 |
| lmacken | so I guess our goal should be to: make it trivial to write tests, and have them automatically get run at the appropriate time. | 15:41 |
| lmacken | I know dmalcom had started hacking on a dispatcher for that stuff.. I need to touch base with him on that. | 15:42 |
| |DrJef| | lmacken, okay that's all i got | 15:44 |
| lmacken | k. Feel free to prod me at any time with any other questions/comments (that goes for all of you). | 15:45 |
| lmacken | If no one has anything else, I guess that's a wrap! | 15:45 |
Generated by irclog2html.py 2.5 by Marius Gedminas - find it at mg.pov.lt!