this post was submitted on
8 points (83% like it)
10 up votes 2 down votes
shortlink:
to anything interesting: news article, blog entry, video, picture...

7
8

How can I simplify the parts of my job I hate (configuration management)? (self.sysadmin)

submitted ago by dev_randomJboss/Linux Admin

I work for a SAAS style company. 99.9% Windows shop, though I'm also a linux guy. A vast majority of our IT resources are spent taking new software releases and merging them with client-specific customizations from previous builds. There has to be an easier way to do this than simply using Beyond Compare to diff the whole build and go through by hand. I'm thinking of some way to compare the previous release from development and the current 'live' version, and apply those customizations to the new build from dev. Is there a tool, or even a way to script this (bash/batch/c#/python/etc.)? If it helps these are mostly java/jboss/tomcat apps.

all 19 comments
sorted by:
best
hot
new
controversial
top
old

[–]DrGirlfriend 6 points7 points ago

Are you using any kind of source control? It sounds like what you are asking for is the ability to branch and merge, which would make a case for deploying from Git.

[–]dev_randomJboss/Linux Admin[S] 1 point2 points ago

Source control has been mentioned before, and we do use svn for certain things, so that's possible, but I'm not sure how it would work as I've only ever used it for change control.

[–]simtel20Jack of All Trades 0 points1 point ago

Look at chef, and managing cookbooks/recipes per-client in revision control. Then you can re-deploy applications with your customizations - automagically.

http://opscode.com

There is a free server, a paid server, a paid service, and the option to not use a server (chef-solo).

[–]Hexodamis a sysadmin 0 points1 point ago

Holymf, they finally have windows support

http://www.opscode.com/blog/2011/12/15/chef-client-installer-for-windows/

http://wiki.opscode.com/display/chef/Fast+Start+Guide+for+Windows

[–]simtel20Jack of All Trades 0 points1 point ago

Yep. And actual staff developing windows support.

[–]Hexodamis a sysadmin 0 points1 point ago

Since its pretty new I would not use it right away, but this I'm watching :)

[–]akashaniSr. Linux Admin @startup #8 3 points4 points ago

What you're currently doing sounds... insane.

I'm a bit confused on why customer specific changes aren't in a branch of trunk that could be merged with trunk to create a new branch when you want to release. This should be possible in just about any reasonably popular source control though git is probably the best at branch merging.

[–]dev_randomJboss/Linux Admin[S] 0 points1 point ago

It's a case of "this is the way it's always been done" -- the original group that started this process left, and I'm the only guy left from the second group that learned from the first. How would one go about something like this? Can you point me in the direction of examples?

Our developers release binary releases, so we would have to run a separate tree -- I'm not really visualizing how it would work.

[–]akashaniSr. Linux Admin @startup #8 6 points7 points ago

This article covers some of the basics and has nice pictures. http://nvie.com/posts/a-successful-git-branching-model/

fwiw, as a sys admin I'd put my release engineer hat on tell Dev that generating proper releases is their problem. When they generate a production release they'll do it for each client with the changed merged. Generating releases based on binary releases (what?) after the fact is solving the problem at the wrong point IMO.

[–]dev_randomJboss/Linux Admin[S] 0 points1 point ago

I've read through this, and I think I understand it, but I'm having trouble seeing where it would fit in. Many of the customizations are in files that change regularly, wouldn't it become a nightmare and we'd basically end up doing what we do now and diffing them by hand?

[–]akashaniSr. Linux Admin @startup #8 0 points1 point ago

If that's the case, then you may need to change to code to accommodate this. Rather than having large files you need to modify, you'd break into smaller files that you could replace completely as needed. Perhaps even better is to create config/custom/ or html/custom (or whatever) which is parsed last so that you can override defaults with your own config files. Lots of different methods, but they are require some sort of entry point on the software side.

[–]dev_randomJboss/Linux Admin[S] 0 points1 point ago

That's why I keep thinking, if I could use the 'default' build from the previous release as a reference, then apply the differences between the live build and it's dev release, to the new dev release, that would be the ideal solution. I'm just not sure how to implement it.

[–]Kinetic_Static 1 point2 points ago

It really sounds like you need continuous integration . You can some pretty amazing things with CI, but it will take some time to set up.

It sounds like if you can the application code into SC, you can then put all the client configs in SC, and have your CI automatically pull the new changes in the application. If it monitors the application SC and the Client Config SC, anytime it sees a commit, it can rebuild the full application for your client. After it's built it can deploy it if needed.

[–]Commod0reUNIX Admin / Software Dev 0 points1 point ago

http://inedo.com/buildmaster/features/continuous-integration Buildmaster can do it as well.

I don't have an amazing amount of experience with Buildmaster, but what I have used of it, it's pretty awesome and powerful at automating all kinds of build tasks.

[–]Kinetic_Static 1 point2 points ago

I've been using hudson/jenkins for a couple of years now. It can be a little piggy, but I love that you can do anything with it. I'm currently adding our Chef infrastructure so when we add something to chef, it automatically creates machines to test the changes. The extensibility is great.

[–]cparedessyseng for the clouds 2 points3 points ago

It sounds like you'll want a combination of a few things:

  1. Artifact management. You're looking at things like Nexus for managing these things (these are things like compiled JAR's or vendor JAR's that you need to use as dependencies.)
  2. More modern source control practices. Use feature branches, hotfix branches, etc. liberally.
  3. Configuration management. After the app is deployed, configure the application automatically.
  4. Continuous integration. Try to get yourself to a point where you don't have to worry about diffing things, as your CI tool should be able to do that automatically.

[–]cornet 1 point2 points ago

Are these just configuration differences or actual code differences ?

[–]dev_randomJboss/Linux Admin[S] 0 points1 point ago

Yes. There are some closed libraries that we don't touch, but generally there's a mix of XML configurations and customized HTML/JSP/images/etc.

[–]cornet 4 points5 points ago

If you're having to manage more than 1 or 2 different versions then, as you've realised, you're doing it wrong.

The solution however is not to start maintaining 10 different branches for 10 customers but is to build the customisation ability into the application itself.

There really needs to be a core application (distributed as a .jar/.war/whatever) then all configuration and customisation needs to be separate.

The extra resources (config files, images etc...) can then be maintained by some configuration management software like puppet/chef/cfengine or you could build the functionality straight into the application.

Think along the lines of a content management system where you have a core application then themes/plugins that can be applied etc... Or something like Microsoft CRM where you can customise via scripts.

Use of this site constitutes acceptance of our User Agreement and Privacy Policy. © 2012 reddit inc. All rights reserved.

REDDIT and the ALIEN Logo are registered trademarks of reddit inc.

π Rendered by PID 2958 on app-120 running 86c9e1d.