sharing my passion about software and game development
I probably won’t be blogging some tech articles this year. So let me say it already:
Merry christmas and a happy new year!
Coming in 2012:
Want to do some cool coding, and you’re nearby Amsterdam? Check out the code retreat I am organizing at 7th of january in 2012! And join it! 😀
Recently I’ve encountered a problem when deploying a web application using our CI environment.
The probl… challange
Sometimes the deployed website was not the latest version (with latest changes).
I hope this blog post can save you some time, while figuring out what went wrong.
A little background
In this case, our CI environment is Jenkins, running on a Windows server. The artifact produced (a WAR) is uploaded to a Linux server where a deploy script is being ran, taking care of shutting down tomcat’s, cleaning work directories, and eventually copy the new WAR, then start everything again.
First checks
First thing I have checked: Do my dependencies compile at the right order? Are they being deployed into our own artifactory? Is the task, responsible for creating the WAR, using the latest dependencies?
All these things where setup ok.
So why does it go wrong?
Deployment strategy
In our case, we use the SVN revision nr as a version for our webapp. It is used by the deployment script, and by the uploading scripts as well. So this means, that when we are deploying revision 1000 of the website, a directory is created like : revision_1000 and the WAR file is uploaded into that directory. After that the deploy script is ran with the revision parameter.
This works well. It is used for several clients, and it has proved its worth thousand times. So this can’t be the problem right?
Branching strategy
There are a few branching strategies you can use within your projects. In this case, there is for each artifact an own trunk/branches/tags. This means that we can develop each component independently from each other. We can also branch versions of components. The website decides which versions of the components it uses.
This has several advantages. This particular website runs against a webservice that could change quite rapidly. We could be developing against version 1.0. Then later it would be 2.0, but when we had to go live we had to switch back to 1.0. Because of the branching strategy we have different branches for each webservice version, this mainly contains generated code from the WSDL’s and a thin layer. The website could easily switch between versions. Since we’re using Maven it was a matter of changing a version number of the dependency in the POM.
So everything went fine, but it still did not work? How is that possible?
Well.. thats where the real digging begins. It seemed that Jenkins reported that the task has completed with succes. However, when looking at the log, the script seemed to have failed. It failed because it could not create the revision folder to upload the WAR file to.
But how is that possible?
In this case, it is because of the branching policy.
The effect of the used branching policy
Given each module a seperate trunk/branches/tags. When a change is made at component A, then it will not automatically raise the revision nr in component B. Only when a change is made at component A again, the revision will be raised there.
Meaning. When website (component A), is at revision 1000. And it uses a dependency (Component B), at revision 999. When deploying the website, a revision_1000 directory is created.
The culprit
When a bug is found in Component B, and it is fixed. This will make Component B revision 1001. However, the website remains revision 1000. When the deploy task is checking out the website repository, it will find that the current revision is 1000 for that trunk. And, it will work from there. The deploy script wants to create a directory revision_1000 again, which will fail and will abort the script. It does use the latest dependency though, but since the creation of the deploy directory failed, it never uploaded the latest WAR (it did not overwrite this).
“The” fix
I see two ways to get out of this situation. Fix the deploy script, or change the branching policy.
Fixing the script…?
Some may argue that it is correct, that the revision remains unchanged for the website, and that the deploy script should know how to deal with this. One way would be to not only use the revision number as deploy directory, but also a timestamp. Ie, revision_1000_DD-MM-YYYY-HH-MM. For instance: revision_1000-12-1-2011-21.
Change the branching policy
Another way of fixing this issue, is to make sure that every time a change is made the revision number is being raised. The only way to be able to do this, is to let every component be beneath a branch or trunk, instead of having their own branches.
The components that are bound to the 3rd party webservices remain their branching policy. The downside would be that any fixes in these components still have this issue, but this time deliberately.
I’ve visited JFall, a conference held by the NLJUG (Dutch Java User Group), at 2 november 2011. In this blog I’d like to share my experiences of this conference. This was my first time at this conference and I pretty much had no expectations.
There was a lot to go to; I have visited the following seemingly interesting topics:
– Keynote – Java 7 Directors Cut
– Overthere – Design and implementation of a Java Remote File and Execution framework
– Hands on lab – Clojure – A gentle introduction to a brilliant language
– Keynote – Building Highly Scalable Java Applications on Windows Azure
– Migrating Spring to Java EE 6
– Hands on lab – Whats that smell? – Refactor away that nasy odor
Its completely personal
For each topic I will write down my personal experiences. They are by no means complete. In fact they are completely biased. If you also have visited the JFall conference and you feel different about one of the topics; let me know in the comments below.
Keynote – Java 7 Directors Cut
The keynote consisted of two parts: The first part was about Oracle; how they are not that bad and how they had a rough year in 2010. All I was thinking was : “get on with it“. The second part was about Java 7. It took quite some time before Java 7 was released. (yes Java 7 is released already). In this part it was explained why it took so long and how Oracle played a role in getting it released.
Overthere – Design and implementation of a Java Remote File and Execution framework
This one was actually quite interesting. The talk was about a remote execution framework. Roughly said you could connect to a remote machine via this API, and then execute commands there. Its main goal is to make automated deployment easier. What I missed is they “why” part. Why is this framework built in the first place? What kind of trouble was this framework meant to solve primarily?
Hands on lab – Clojure – A gentle introduction to a brilliant language
Although the start of this session was a bit clumsy; it was really hands on. We did a lot of practice stuff, but also here I did not get any answer to why and when I should use Clojure in the first place. Someone asked if it was used within webapplications. But it seems it is not used that much at all within webapps. (is this so?) The language itself is something to get used to, and 50 minutes was really too short for this. It did spark interest a bit. What I found most intriguing is that using Clojure it forces you to think differently about solving problems. I found that very valuable.
Keynote – Building Highly Scalable Java Applications on Windows Azure
Microsoft presenting at a Java conference, who would have thought that? I think this actually was a great way for Microsoft to show how their attitude towards the Java community has changed. And for good reason. There are tons of Java powered (web) applications and Windows Azure is a platform capable of running them.
The keynote was given at a high speed; and it had (very daring) a live demo. Of course something went wrong, but in the end things did work. Also everybody could get a 30 day free trial of Windows Azure. The presentation concentrated about the effort needed to run Java based applications into Windows Azure. Also, it showed what configuration was needed to actually deploy an application. Aside the Java ‘support’ at Windows Azure, some architect schema’s where shown of well known websites (Flickr, Twitter, etc) and the match with Windows Azure was made. (Like: Yes you can do all these things with Windows Azure).
Migrating Spring to Java EE 6
This show starts with a big disclaimer, in the form of 3 slides and lots of talking about that “this presentation is not a shoot out”. I was wondering why, but the reason became clear after a few slides after that. In short, it was all the way “Spring Bashing” all the time. The use-case was that there was an old Spring application (what is old anyway?). And that you somehow must migrate. You have two options:
A) Migrate to a new Spring version.
B) migrate to Java EE 6.
Since we both know the subject of this presentation we are *not* choosing option A. Then the ‘fun’ part begins. The reasons given to use Java EE is mostly “because it is the standard”. I always get the chills of such comments. What is the standard anyway?
Its a pitty, because the reason that Spring is propriety software is actually valid. However, it just won’t cut it when yelling “its the standard“.
Then it begins, the migration path. And guess what the first step is: Update to the latest spring version. The reason is that Spring has a great backwards compatibility. One would wonder why to migrate further away from Spring from that point. Basically migrating to the latest Spring version would be easier, painless and won’t cost your business that much.
Something different that struck me was an example of how Spring is unable to render objects given by a (JDBCTemplate) DAO as you will be presented with a LazyInitializationException. So the solution in Java EE is that you put an anotation that says the bean scope is “Request”. Apart from the fact that Spring can do this too, it is also in my opinion (in Java EE and in Spring) a bad practice. I think that objects returned by DAO’s should *never* be used in the view. They are not the same thing, so don’t even try it. If you do you’re being lazy.
I think the presentation would be much better if the starting point would be an assignment like “build a web application”. Then you could argue if you would need Spring and why not (since it would be about Java EE 6). Then, you could show everybody how Java EE 6 is capable of doing things Spring can do and how much it has been improved. This would probably be much better than bashing into an existing (older) Spring application and telling everybody to migrate just for the sake of standards. I don’t think any customer would be willing to pay that bill.
Hands on lab – Whats that smell? – Refactor away that nasy odor
I love refactoring, so this one was more of a nice way to close the day. To me it was more like a rehearsal and getting the vocabulary right. I actually found that quite a positive side. Too bad it only lasted 50 minutes, which was way too short. The introduction was also a bit too long so the actual development time was not that long.
In overall
It was big. It was busy. It was very nice to experience. Often I wanted more, more in depth, more time. I see that as a good thing :). On the other hand, I think the NLJug should consider to spread JFall over two days. This way people can choose between several sessions, and the sessions given can be longer and allowed to be more in depth. Especially if the program has some repitition in it, I could choose to visit some workshops on day one, and then watch the presentations at day two which I would have missed at day one.
Recently I have been experimenting with a Code Kata, and in this post I’d like to share my experiences with it.
Code Kata’s have been around for a while, but it really came into my attention while reading Chapter 6 from the book The Clean Coder by Robert C Martin. This chapter makes an anology that at your work you’re a performer like a musician and outside work you (should be, like a musician) practicing. (Of course, you will learn while at work, but that is not the point).
I have played the piano for around 6 years (followed lessons) and played it much less after that (in fact, I don’t really play at all anymore). In those six years I had to practice etudes as well as more famous pieces. I did not understand why I had to practice Etudes, until much later.
So what has an Etude to do with Kata’s? Lets look at the description of an Etude (at wikipedia):
“an instrumental musical composition, most commonly of considerable difficulty, usually designed to provide practice material for perfecting a particular technical skill”
Without going into detail of a Kata itself, it is used to practice and perfect a set of techniques. Repetition and practice until you’re able to perfectly perform a Kata (or an Etude if you will) will help you further when you need to improvise or apply it in different forms. A lot of Etudes, techniques, are used in real pieces. A lot of Code Kata’s are actually dealing with real world problems.
So if I practice enough code kata’s I will become good at any problem I might face when writing software?
Not quite.
Code Kata’s are flexible; meaning you must set yourself a goal you want to achieve by doing a kata. When doing an etude you don’t have a lot of options. Your main goal is getting better with your fingers to play a series of notes or transitions. With a Code Kata you could practice your typing. Or perhaps practice all short-cuts of your IDE. Or heck, learn a new IDE while doing one. Perhaps you want to learn a new language. Perhaps you want to get better at TDD. Or you simply want to get to the solution and find the most efficient way to do so.
Atleast, I found doing a code kata much more fun than doing an etude 🙂
The Bowling Game Kata is a Kata that challanges you to write a class (Game) that simulates a bowling ball game. You can roll balls and give the amount of pins knocked down. At the end you can call the score() method and you should get the correct score. It takes all rules into account, gutter, spares, strikes and the tenth frame where you can have 3 rolls instead of 2. The perfect game has 300 points. The worst game 0.
My initial thought: How hard can this be? I mean come on, I’ve dealt with harder things than a bowling game scoring system. Since I did not do any Code Kata before I set my goal to find the solution to this kata while doing TDD. I also did not want to look too much ahead in Uncle Bob’s (very nice) presentation (with solution). So I stopped when the game interface was given (roll() and score()) and I went ahead. Again, how hard could it be?
I have tried this Bowling Game Kata three times, and for each attempt I have written my experiences. All in all it was a very good experience and I recommend to try it out yourself. I believe if you want to get better, you need to practice. And only if you tried this multiple times, only then you know how it is.
So instead of talk the talk, let me walk the talk…
Goal: Get it working, while doing TDD.
I set up a simple project and started with the GameTest. The first two tests where easy to do (0 pins, and all ones). But as soon as I got into Spares my first thought was to create a Frame class. Because a Frame represents a ‘turn’ where you can only roll 2 balls. The Frame class was born, along with its unit test. And I thought it felt good. I even added more ‘cheating’ detection. So you cannot roll 3 balls in one frame, or you cannot say you rolled 2 pins and then 9 (making 11 in one frame). I felt great and got unit tests working like nothing could stop me. Until the ‘perfect game’ test came around and my model just fell apart.
There was no way I could make it fit, without bending my entire model/solution.
So there I was, totally excited and thinking “just one more test and I’m done”.. and I got this.
Eventually I fixed it, I made my Frame class more flexible so I could set the maximum rolls and added flags. I then could create a TenthFrame class and set its flags so it would score differently. I also had created tests for the TenthFrame and even used an Abstract test class so I did not have duplicate code. Even so, I felt like this was wrong. I was bending my design just to work for one exception in the rules.
When I got all my unit tests passing, even the perfect score game, I just felt a great deception. My design sucked. Also, it took me almost 4 to 6 hours to get it working. Way too long for a code kata right?
Lessons learned
– TDD cycles where not strict enough; so…
– TDD cycles where slow, I had to switch mouse/keyboard to rerun the unit test(s)
– I made design decisions too early and later got ‘stuck’ and had to bend the design to make it work completely
– Finding the solution the first time takes time
– I made much more stuff than I had to (over-engineering?)
Trivia
– Time taken: roughly 6 hours.
– Amount of unit tests: 33
Goal: Get it working, while doing TDD. Take lessons learned from first attempt. Aka: Tighter TDD cycles, get faster at TDD cycles, etc.
I started this kata late in the evening. I spent a fraction of the time compared to the first attempt: ~ 45 minutes(!). The later half hour mainly refactoring and keeping green bars. The actual solution was there within an hour. I did not have to bend my design, I could keep everything within the Game class!
Something little, but practical I learned about the IDE I used (Eclipse) is to short-key the ‘rerun last test’, so my TDD cycles where shorter.
I also noticed I understood the scoring of the bowling game much better. I don’t play this game very much, and when I do, the computer does all the scoring for me. So I guess the first attempt at the Kata took also longer because I had to understand the scoring rules.
Design wise I found that I still use “frames”, but not as a separate class. I do not have any cheating detection (so I can roll 12 pins and it won’t complain), but the scoring will utterly fail in that respect. Building these checks in would not be a problem though, because the spare/strike detection is now so easy.
Lessons learned
– Faster TDD cycle by using shortcut for re-running tests
– TDD cycles where stricter, but could be even more tightened
– Commenting out tests when more than one breaks really helps you get focussed on getting one thing to work (so leave one test breaking), instead of fixing all tests at once.
– No cheating detection, no over-engineering
– Of the time consumed, I spent more time refactoring relatively to the first attempt, than thinking / finding the solution.
– I could refactor a lot of code, and make it much more cleaner. And safely due the tests.
Trivia
– Time taken: roughly 45 minutes (!!)
– Amount of unit tests: 7
Goal: Tighten the TDD cycle. Get it done in 30 minutes or less.
This time I started fresh on a Sunday morning. Since I know the solution and I knew the design choices I made earlier (I know what works, and what does not work) things went very quick. I finished it within 30 minutes. I had the same amount of unit tests and the greatest thing was that the last test (perfect game) worked immediately. I did not had to change anything to make it work.
Once I had the last test working, I checked the code a bit and called it done. This time I also wanted to check my solution against the original presentation Uncle Bob made, to see if I missed anything or not. I figured that some of my tests where faulty:
– If you roll only ones, you can only roll 20 times and not 21 times. (i had a weird if statement to fix this up, but now it seemed that this was flawed).
– The perfect game was in my case 21 strikes, while you can only roll 12 times in that case. When I changed my test, it still worked.
It struck me that I was approaching this technically (21 rolls is maximum), and not from a functional point of view (ie 20 is max when only ones).
I also found that my TDD cycles where still to wide. I could do run the cycles close to each line of code, but I tend to write 2 or 3 lines before re-running my tests. Especially the first tests where suffering from this, later tests went better.
Lessons learned
– TDD cycles can be shortened
– Functional point of view caught errors in my tests
– Shortest kata ever (under 30 minutes)
– Latest test worked, design was good. Design was even better after fixing the test for ‘only ones’, so I could remove weird if statements.
Trivia
– Time taken: < 30 minutes
– Amount of unit tests: 5
So, after 3 attempts, do these Code Kata’s work for me? It surely learned me a few lessons. In short:
– The first time you do a Kata, it is slow. And you’re focussed on the solution. This could throw you off, but you have to persist…
– Later attempts are going much faster, and your focus shifts to other techniques. Mine was mainly speeding up my TDD cycle.
– I have learned a few things to speed up my TDD cycle, which I can apply in real world stuff as well. Which is good!
– Over-engineering will bite you, one way or another. Your design will be toast.
– I have learned that even in my last example I was approaching the solution too technical. As a Developer I still did not approach it entirely functionally. This meant that even though I thought I was done, I wasn’t. I do plan to use Acceptance tests (using JBehave) for this. That will be covered in a next blog.
– Above all, doing a Code Kata is fun!
I would advice to other developers to do Code Kata’s and get better at what they are doing. There are tons of areas where you can improve. In short, yes I do believe in them and I think you should give them a go, if you haven’t already!
Find more about Code Kata’s:
http://codekata.pragprog.com/
http://stackoverflow.com/questions/44533/your-favorite-code-kata
http://www.codinghorror.com/blog/2008/06/the-ultimate-code-kata.html
If you really want light-weight warm up exercises, you might want to go to: http://codingbat.com/
In one of my previous posts I blogged about refactoring I did for my hobby project.
Robert C Martin has a nice video showing refactoring, using IntelliJ. Watch it here.
You can also see why unit testing is so important, as with each change the unit tests are being ran in order to check if anything got broken.