Software, Mathematics and Money

Static excursion aka GPS Noise

2011-05-19T05:07:00.001-07:00

Two views of the same phenomenon. Aka noise

500751504,A,110519063636,N5121.3833W00011.1017,000,247,NA,17800000,108
500751504,A,110519063606,N5121.3835W00011.1007,007,247,NA,17800000,108
500751504,A,110519063536,N5121.3832W00011.0866,000,074,NA,17800000,108
500751504,A,110519063506,N5121.3706W00011.0796,000,345,NA,17800000,108
500751504,A,110519063448,N5121.3673W00011.0927,010,086,NA,17800000,110

VS2010–Flaky IDE. I don’t want to send any information, I just want an IDE that doesn’t crash every 30 minutes.

2011-05-01T14:52:00.001-07:00

You have to setup and tear down

2011-04-16T15:32:00.001-07:00

Recently I witnessed the execution of performance test runs that did not include the elements necessary for successful and reliable result generation.

The test run should have included the following activities:

Test setup

Seed the database with fresh data
Rebuild indexes
Recreate statistics

Test execution and
Test tear down

Delete data generated during the test
Clear caches
Recycle application pools
Remove all database deadlocks

The initial performance run included steps 1-2; subsequent runs only included step 2. When the results from all runs where compared, it was observed that the performance of the system under test(SUT) gradually deteriorated.

The main point is that without proper tear down and setup after each performance run, the tests were measuring a different SUT each time.

For performance comparisons to be meaningful, one has got to compare similar SUTs.

If you can’t perform steps 1-3 because of complexities in your environment, then abandon your performance tests until these prerequisites are resolved.

Google Chart API

2011-04-05T13:52:00.001-07:00

We are building a lightweight reporting module; our charts need to scale and given that we could be hosting the report module on a nix platform, using the .NET charting APIs was not an option.

After a bit of digging we settled on using the Google Chart API which provides a POST API. All we needed to do to perform integration was supply the data series, labels, chart type, chart dimensions etc.

https://chart.googleapis.com/chart?cht=p3&chs=300x100&chl=Agribusiness|Haulage|Jobs|Other|Property&chd=t:6,3,2,2,1

The rendered chart is displayed below

This is a great services from Google and demonstrates the power of software as a service(SaaS). Software development should be about assembling and orchestrating various services to build a cohesive feature set.

Zam-Track on track

2011-04-03T09:09:00.001-07:00

Based on current data volumes and interest expressed in the Z-Track platform, we expect to process about 1 million data points this year. Each data item has the following structure:

%%800704,A,110402201650,N5121.3489W00011.1780,000,230,NA,47000000,531,CFG:Z31,10,1|

A JSON representation of this feed looks like this:

ID:800704,GPSValid:A,DateTime:110402201650,Loc:N5121.3489W00011.1780,Speed:000,Dir:230,Temp:NA,Status:47000000,Event:531, Message:CFG:Z31,10,1|

The structure of the data makes it ideal for a document-centric data store. We have evaluated MongoDB as the storage solution; other contenders in the running are CouchDB. Fast data writes and reads are essential for our application.

The feed gives us vast opportunities to integrate with HR, Payroll and financials systems.

Quick time to market with DTSTTCPW

2011-03-15T18:27:00.001-07:00

We are developing all our products using this development philosophy with the caveat that we then “refactor mercilessly”. So far this has helped us to deliver key application features in record time. The success of this approach relies in having the discipline to rapidly payoff the technical debt that inevitably accrues.

And of course as with everything – context matters. Simplicity is relative.

reCaptcha

2011-03-15T18:06:00.001-07:00

Is pure genius and makes you think about how the solution to one problem, Spambots, can provide valuable commercial opportunities in another totally unrelated area. Great idea! But do the deciphered publications end up in the public domain or will Google charge us for this material?

Integration of reCaptcha is quick and painless, so we are adding it to all our online form submission workflows.

Just Tether It

2011-03-15T17:50:00.001-07:00

So my mobile phone provider says I can eat as much data as I like on the current tariff; the truth of the matter is that I don’t really surf the web whilst I am out and about. At the end of my billing period I have tonnes of unused bandwidth. Luckily I have managed to download a USB tethering utility, PdaNet, that allows me to use my phone as a wi-fi hotspot via USB cable. Sweet. Let the feasting begin…

Fleet Management Platform codenamed Z-Track now in development

2011-03-13T16:00:00.001-07:00

Zainco Ventures is developing a fleet management and asset tracking platform, Z-track , aimed at providing deep integration with back office systems. The first beta release is planned for Q3 2011.

Chisamba order confirmed

2011-03-13T15:16:00.001-07:00

After a difficult trading period in February, business is definitely picking up. Zainco Ventures has been engaged again by the Food Reserve Agency to help with the movement of 300 metric tonnes of vital grain stocks. This is a ringing endorsement of the services provided by the company. Last year Zainco Ventures helped the Zambian government to move over 4000 metric tonnes of grain to safe storage.

With a forecast yield of 2.8 million metric tonnes in the 2010/11 agriculture season, we will definitely be busy.

Reviewing the £100 Android tablet

2011-03-13T15:10:00.001-07:00

There is a tablet for almost any budget. My tablet arrived 1 week ago and since then I have been evaluating the wowPad.

I have successfully upgraded the firmware using a download from the internet. Everything works as it should.

The camera is low spec, but this is to be expected for a budget tablet.

Overall – I am quite impressed with device. Would I buy more units? Definitely yes!

Description Resource Path Location Type error: No resource identifier found for attribute 'onClick' in package 'android' main.xml /HelloFieldroid/res/layout line 9 Android AAPT Problem

2011-02-08T12:07:00.003-08:00

Appears there is a problem with using events declaratively, so have ended up implementing the event handler in code

An SDK Target must be specified when you try to create a new Android Project in Eclipse

2011-02-08T12:07:00.001-08:00

This is caused by the SDK path not being set in the Eclipse IDE. This is resolved by selecting Window—>Preferences—>Android and enter the path to your Android SDK, then hit OK. You will see a list of all your Android Platforms as shown below:

   1:  <?xml version="1.0" encoding="utf-8"?>

   2:  <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"

   3:      android:orientation="vertical" android:layout_width="fill_parent"

   4:      android:layout_height="fill_parent">

   5:      <TextView android:layout_width="fill_parent"

   6:          android:layout_height="wrap_content" android:text="@string/hello" />

   7:      <EditText android:id="@+id/editText1" android:layout_height="wrap_content"

   8:          android:layout_width="fill_parent" android:autoText="false"></EditText>

   9:      <Button android:layout_height="wrap_content"

  10:          android:layout_width="wrap_content" android:id="@+id/button1"

  11:          android:text="save note"

  12:          android:onClick="saveNote"

  13:          ></Button>

  14:  </LinearLayout>

Monodroid vs Eclispe with ADT

2011-02-08T12:06:00.001-08:00

For 400USD looks like Monodroid is definitely an expensive way for .NET developers to join the Android game. Using Eclipse and the Android Development Toolkit(ADT) an experienced .NET developer will be productive in no time. I downloaded the tools 1st thing in the morning and by midday I had a simple data driven application with the appropriate test coverage.

The Eclipse tool support for Android is unbeatable. You get design surfaces, inbuilt editors for resource files and the comfort that these tools are mature and bug free. The other side effect is that you end up learning Java.

Viva free tools!

Lift off!

2011-02-05T14:36:00.001-08:00

Monodroid resource generation fails

2011-02-04T19:05:00.001-08:00

------ Rebuild All started: Project: Com.Fieldroid.Search, Configuration: Debug Any CPU ------
C:\Program Files\MSBuild\Novell\Novell.MonoDroid.Common.targets(281,2): error MSB6006: "aresgen.exe" exited with code 1.
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

Error 1 "aresgen.exe" exited with code 1. SmsSearch

2011-02-04T18:56:00.001-08:00

Another Successful release... times are good

2011-02-04T03:35:00.001-08:00

We STP Heroes J - YES, We have done it AGAIN! J Another milestone success!!

We hear from Business that Sprint 13 Release in PROD is doing great!

Thanks to all for all the hard work on this one other major milestone release that has gone LIVE this morning.

Celebrations… we are just shot of balloons – that is all …J

I have brought in a little Sainsbury's Sweet shop, yes really!!!!!

Keep up the nice work J

Grab the sweets before they vanish. I am running a TPL Parallel LINQ query to eat all sweets….

// A little celeberation in my style… ;) with C#

// Sundar: Start diving into the sweets… use TPL so I can eat ALL, FAST… J

// But of course, I am running an i7 980X 12 MB cache, with Six-cores… hyper-threaded to 12 virtual cores, mind you…. wohoooo……..

Parallel.ForEach ( sweetsOnMyDesk, oneSweet =>

{

// The more sweets I eat here, the greater the speed compared to if I used a non-parallel foreach loop for my eating process J

var sweet = CeleberationsHelperUtils.UnPackSweet(oneSweet);

this.Eat(sweet); // I really wanted to say Me.Eat(…) but that would sound like VB.NET ;) Grrrrrrr……… I am "C# house", baby! J

}

// Wow!! what a sweet C# lambda expression I have got there – don't you love it? J

);

// Execution done!

// Let others know I am eating… very fast… using Parallel extensions ;)

Console.WriteLine("Sundar has completed eating ALL (?) sweets. Press any key to let me continue to work on Sprint 14 J.");

Developing WCF Services on a locked down corporate build

2011-01-11T08:25:00.001-08:00

Software archaeology

2010-11-06T08:09:00.001-07:00

A recent addition to our portfolio of problems is a widely successful application that has been developed by business users to process commission payments.

The application has largely been built by the client over a period of time and continues to be useful to this day. A change in the operational landscape means that this software won’t scale beyond its current functionality and processing capacity. An upgrade to a full-scale enterprise system is a must rather than a luxury.

As is the case with such organic systems, the entire concepts of the domain are embedded in functions with lines in excess of 1000. You start off following the code and break pointing every significant statement, and before you know it you have lost your bearings. The questions then becomes - “what was the intention of the civilization that built this technology”; with curiously named variables and loops nested to a depth of 5+ deciphering the intention is non-trivial. One has to methodically piece together all the inputs, configuration data and perhaps chant some voodoo spells to get the system to work.

UML Interaction diagrams seem to be a powerful tool in understanding the runtime intention of legacy software, but the subtlety contained in conditional statements can be quite problematic to document. It is tempting to ignore the body of knowledge embodied in the legacy application, but doing so makes the reverse and forward engineering unnecessarily complex. So like the archaeologist sifting through the fossilised remains of a once flourishing civilization, I crack open the IDE and good old NotePad in an attempt to gain understanding.

A good coffee from back home...Zambia Terranova

2010-10-18T11:28:00.001-07:00

I have been contemplating going back to Soho to get my coffee supplies replenished, but this rare find in Sainsburys saved me the trek to central London. The taste is not as sour as I would like and it has not got the heavy taste I have become accustomed to. Anyway I would give it a 3/5.

A delightfully parallel problem

2010-10-17T08:55:00.001-07:00

I am currently recruiting and part of this process involves the analysis of lots of CVs. This can be quite time consuming, so I have decided that I will expedite the process by developing a small text mining application that analyzes the resumes and produces a signature for each document. The most commons words in the CV will then be readout by the computer and if I like the sound of the digest I will shortlist the candidate.

The essential parts of text mining are being able to tokenize the document and filtering out noise words. Thanks to PLINQ, I can do the following:

PLINQ clusters words in doc
 private static IEnumerable<IGrouping<string, string>> CalculateWordFrequency(string[] content)
{
    var groupedWords =
        content.Select(word => word.ToLowerInvariant()).GroupBy(word => word.ToLowerInvariant()).Where(
            word => word.Count<string>() > 2);
    return groupedWords.AsParallel();
}
 

The document tokenizer approach is fairly naive and could do with more work.

After we have grouped the words in each resume, we then use the Microsoft Speech API found in the namespace, System.Speech.Synthesis to recite the most common terms

Reciting the most common terms
 public static void ReciteResume()
{
    using (var speechSynthesizer = new SpeechSynthesizer())
        foreach (var item in GetGroupedTermsInResume(FilterContent(TokenizeContent(ReadDocument()))))
        {
            speechSynthesizer.SetOutputToDefaultAudioDevice();
            speechSynthesizer.Speak(item.Key);
        }
}
 

I have already hired a dozen plus developers using the traditional filtering approach, so it would be interesting to see how the results of the automated CV selection process compare.

An alternative method of representing the CV digest is to generate a logarithmic plot, “signature file” as shown below. I wonder what signature represents an ideal candidate. I would probably need to analyze large amounts of data to arrive at an empirically valid conclusion.

Incidentally this candidate was not hired as their CV contained a lot of buzzwords and they could not explain how they had used the technologies.

And this is what it sounds like [audio file].

You gotta love Design By Contract

2010-10-09T11:04:00.001-07:00

Code Snippet
 [TestMethod]
[ExpectedException(typeof(TradingServiceException))]
public void ShouldThrowExceptionForInvalidTrade()
{
    var mockTrade = new Mock<AbstractTrade>(MockBehavior.Strict);
    mockTrade.SetupAllProperties();
 
    mockTrade.Setup(trade => trade.IsValid()).Returns(false);
 
    var tradeManager = new TradeManager(mockTrade.Object);
    tradeManager.Execute();
 
    mockTrade.Verify();   
}
 

And when the test fails, we get the following message:

TradeGeneratorTests.ShouldThrowExceptionForInvalidTrade : Failed

Test method Zainco.Commodities.Unit.Tests.TradeGeneratorTests.ShouldThrowExceptionForInvalidTrade threw exception:
Zainco.Commodities.Exceptions.TradingServiceException: Precondition failed: Trade.IsValid() == true Trade execution failed

at System.Diagnostics.Contracts.__ContractsRuntime.Requires<TException>(Boolean condition, String message, String conditionText) in :line 0
at Zainco.Commodities.TradingService.TradeManager.Execute() in TradeManager.cs: line 28
at Zainco.Commodities.Unit.Tests.TradeGeneratorTests.ShouldThrowExceptionForInvalidTrade() in TradeGeneratorTests.cs: line 36

Sweet!

CodeContracts break encapsulation and Resharper 5.0 is blissfully unaware of them

2010-10-09T10:54:00.001-07:00

Seems I have to make my helper methods public if I want to use them within a code contract, but I avoid this by defining a property with the private setter to end up with the Contract.Requires<TException>(…) implementation below:

Code Snippet
 public AbstractTrade Trade 
{
    private set { _trade = value; }
    get { return _trade; }
}
 
public void Execute()
{
    Contract.Requires<TradingServiceException>(Trade.IsValid(),
                                               "Trade execution failed");
    if (TradeExecutedEvent != null)
    {
        TradeExecutedEvent(this, new TradeEventArgs(Trade));
    }
 

C:\Sandbox\Pricing\CommodityServer\TradingService\TradeManager.cs(24,13): error CC1038: Member 'Zainco.Commodities.TradingService.TradeManager.get_Trade' has less visibility than the enclosing method 'Zainco.Commodities.TradingService.TradeManager.Execute'.
C:\Sandbox\Pricing\CommodityServer\TradingService\TradeManager.cs(24,13): warning CC1036: Detected call to method 'Zainco.Commodities.Interfaces.AbstractTrade.IsValid' without [Pure] in contracts of method 'Zainco.Commodities.TradingService.TradeManager.Execute'.
elapsed time: 294.0169ms

Are all circular dependencies created equal?

2010-10-08T15:22:00.001-07:00

I am using the separated interface pattern to implement a commodity trading engine for a bourse in the emerging markets. My unit test package is mocking one of the interfaces and consequently has a dependency on the interfaces package. Likewise the Service package depends on the interfaces package.

Superficially I appear to have a circular dependency and eager Resharper 5.0 complains about this with a fairly descriptive error message “Failed to reference module. Probably reference will produce circular dependencies between projects.”. The result is that intellisense breaks!

What to do?

Are all circular dependencies created equal? Does it matter that the offending dependency in this case is an abstraction rather than a concrete type?

On further examination, the only real dependency is between the test package and the trading service, the other dependencies essentially enforce the contracts between the interface package and those that must either implement the behaviours defined by this package or use the behaviours provided by these contracts.

Tools hmmm….

Today I learnt about A/B testing and applied it to an optimisation problem

2010-08-05T04:04:00.001-07:00

Learn and apply…

Refreshing Exceptions…

2010-07-29T15:20:00.001-07:00

Ever faced the cryptic exceptions that leave you red-eyed from looking at the debug window and inspecting automatic variables? Well if you have spent most of your adult life making money from software development, the probability of this happening is fairly high. So when you encounter an exception message like the one below, you kind of know that the software/API has been developed by developers that care:

WorkOrderTests.ShouldSaveWorkOrderDocument : Failed

Test method WorMaSysUnitTests.WorkOrderTests.ShouldSaveWorkOrderDocument threw exception:
System.InvalidOperationException: The maximum number of requests (30) allowed for this session has been reached.
Raven limits the number of remote calls that a session is allowed to make as an early warning system. Sessions are expected to be short lived, and
Raven provides facilities like Load(string[] keys) to load multiple documents at once and batch saves.
You can increase the limit by setting DocumentConvention.MaxNumberOfRequestsPerSession or DocumentSession.MaxNumberOfRequestsPerSession, but it is
advisable that you'll look into reducing the number of remote calls first, since that will speed up your application signficantly and result in a
more responsive application.

at Raven.Client.Document.InMemoryDocumentSessionOperations.IncrementRequestCount()
at Raven.Client.Document.DocumentSession.SaveChanges()
at WorMaSysUnitTests.WorkOrderTests.ShouldSaveWorkOrderDocument() in WorkOrderTests.cs: line 58

Playing with the Raven

2010-07-29T13:00:00.001-07:00

Zainco is currently evaluating NoSQL approaches for a groundbreaking application we are developing for a security client. At present Raven is looking very promising.

Blogging from new Nexus device

2010-06-05T01:43:00.001-07:00

It's the Epsom Derby today.

30 sprints later…

2010-05-18T13:42:00.001-07:00

I have spent the best part of the last 1 year on a large project with many cross functional teams building a .NET application that integrates with a specialised vendor ERP system. As with any experience, you learn a great deal about yourself and others.

My greatest lesson is that there is a lot of crap software out there making lots of money and that good software (read this as open source) rarely, if ever, makes money.

With the crap software come the zealots and snake oil salesmen selling their bogus cures and panaceas .

If ever I had to express this in mathematical terms, I would say that:

The rate of return on investment on a software asset is inversely proportional to the quality of the code deployed.

This appears to tie with the empirical data, but as with any such observations, outliers or exceptions will exist.

I suppose our challenge as software craftsmen is to ensure that quality and ROI is balanced.

Overheard - "Agile ain't working, we need a plan"

2010-01-31T15:23:00.001-08:00

Commentary to follow

Composite Applications Framework

2010-01-28T12:15:00.001-08:00

Testing the blogging service via Yahoo mail

2010-01-23T12:10:00.001-08:00

2010-01-22T02:34:00.001-08:00

Eurotunnel...

2009-10-03T13:21:00.001-07:00

The hiatus is over; in death is renewal and the realisation that we are indeed mortal.
One
Anyway back to the subject of this blog, software.

At present we face a number of software engineering challenges amongst these is the centralisation of field application logs(FAL). SaaS appears to be a natural solution to this problem. The service interface would ideally accept a log message and persist this for later consumption by development and support staff.

Stuck in traffic

2009-08-21T15:01:00.001-07:00

device drop

Some thoughts on home grown code gen

2009-08-19T16:56:00.001-07:00

The context: I have some xml that needs to be converted to C# class files. The xml is attribute-centric.
Possible implementation: transform using xslt, tokenize xml and use tokens to populate template; use Codedom ap; use code gen tool
Preferred option: CodeDom/Tokenization strategy

Applying Scrum in a non-software development context

2009-08-19T16:45:00.001-07:00

My experience of agile project management is confined to the field of software development, but as luck would have it, I have an opportunity to apply agile in an industry sector where brawn comes before brains. Well that's not entirely correct, but I guess this domain thrives on a command control type structure. I will definitely need to adapt the methodology to take into account the cultural differences afterall we are dealing with complex adaptive systems...ahem

Oh so this is SAP

2009-08-19T16:35:00.001-07:00

Recently started working with SAP as a data model for our .NET UI. All very exciting at the moment. Objects are modelled using a tool and each instance of an object is stored in a construct called a table. Associations and aggregations are aslo modelled using the tool. As someone once said "it's dejavu all over again". Based on these observations, I posit that SAP development exemplifies DDD. The ubiquitous language is developed as the model is formulated, dependencies amongst objects are also modelled accordingly. What I am yet to encounter is the notion of value objects. Entities are an intrinsic part of object definition. Blogged from me Android...

That's your spec sir...my first foray into the world of haptics

2009-08-19T16:10:00.000-07:00

What we have here is a meal planner structure which can be digitised and form a module in our Home Information Management System(HIMS). In this era of the Smart Meter and the Media Center, there is definitely a place for HIMS.

220 Downloads later…

2009-07-26T13:26:00.001-07:00

A while ago I put together a little library that determines internal rate of return for a series of cash flows. This is a problem in the accounting domain and was prompted by the inadequacy of the features in excel. I did not want to haul the whole of excel into the application I was building. I tend to stay clear of Interop if I can help it.

Well the short of it is that the lib averages 1.5 downloads per day. Not much for 15 mins work, but this has led me into thinking that perhaps there is a larger market for reusable math/scientific libraries. How about a DNA sequencing lib?

Now clean it up…

2009-07-26T13:15:00.001-07:00

In our attempt to clean up software engineering and make it a more rigorous profession, I assert that testing is one tool that will ensure that the discipline produces consistent results and repeatable quality. Testing should be at the forefront of a developers’ mind, but in most cases it is an after thought. I suppose this is a reflection on the human condition. We are forever optimists hoping that our artefacts will stand the test of time.

Well I don’t live on hope and for me life is a set of precise constructs. Gravity always wins and code breaks. So lets get test-infected.

Writing a good tests assumes:

A knowledge of the domain or access to a subject matter expert – can be the product owner
An adequate tests framework
A testable architecture

As someone once said, “any one can write a programme, it takes a disciplined developer to write code with tests”.

Balancing the risks of tool switching – a heuristic

2009-07-10T15:08:00.001-07:00

When/why should a team adopt/change its toolset?

If the current toolset is inadequate and results in a loss of productivity
If the new toolset is a ground-breaking innovation and not merely a derivation and solves the business problem better than the incumbent
If there are compliance/regulatory constraints that mandate the change – for example Sarbannes-Oxley.
If the new tools will give you significant competitive advantage in the market place, otherwise the costs of learning/deploying the tools will erode marginal benefits

When should a team not adopt/change its toolset

Because it will look good or make the developers appear smarter when included on their CVs – remember the customer has to deal with mess once you have left the site

The day we overloaded the scrum with issues that were not part of the “triumvirate”

2009-07-10T14:46:00.001-07:00

The context: A distributed team with very minimal inter-personal contact. Most conversations are by electronic means.

The scrum started out with each engineer being asked the usual trio of questions. The expected responses were given and then a tangential conversation about the best approach to take for a new project took root. Big mistake! Before long we had a virtual bun-fight that risked creating permanent schism within the team.

The lesson to learn from this saga is that face-to-face sessions are best when discussing issues bordering on religion. Once a jihad has taken root within the team then delivery of quality software will be impossible because the team is working against itself. As a facilitator it is tempting to give additional bandwidth to groaners and the loudest shouters, but this has to be weighed against the greater needs of the team to work as a cohesive unit.

We have survived the death march…what’s next

2009-07-01T13:22:00.001-07:00

After the intense and somewhat heroic efforts by the team, a retrospective is definitely apropos. The key lessons learnt are:

Scrum definitely works when used in a chaordic context. What might appear as the ritualized trio of questions, what did you do yesterday, what do you plan to do today, what are the impediments to progress, focuses the development team on what needs to get done.
Working software is the ultimate measure of progress on any software project. If it’s buggy you will have trouble gaining the trust and required acceptance and signoff.
Short feedback loops via daily stakeholder meetings help focus the development team on what needs to get done
Having a developer who also happens to be a subject matter expert will propel the team further.
Don’t change the domain model if it’s not broken. A distinction has to be made between purely technical changes and changing the semantics of the model.
Regression issues will undermine the team and erode user confidence
Make your tech choices before hand and adhere to them
Beware of feature envy - http://c2.com/cgi/wiki/Wiki?FeatureEnvy
Keep the testers close and the customers closer
Retrospective testing of code is much harder than TDD.

A severe case of coders block

2009-06-11T15:43:00.001-07:00

With an impossible deadline on my back and a tangle of excel spreadsheets as my spec, I punch the keyboard, but the juice fails to flow in the IDE. The first test says nothing about the problem I am solving. The coffee is insipid; the Ipod sounds like a broken digital record…perhaps the copy and paste coders are on to something…but I let that thought pass. Why bother with reuse if they refuse to use my API. Do I detect NIH?

Making a case for dogmatic Extreme Programming

2009-06-11T15:37:00.001-07:00

Refactoring is now a common term in the developer’s vocabulary. However what most developers call refactoring is really hacking. Making a change to the code in the hope that it will work is somewhat analogous to smoking a cigarette whilst filling up your petrol tank. Both will inevitably end in tears. And this is where the dogma kicks in; it is not really refactoring unless you have a safety net of automated tests to verify that the state of the universe has not changed since you last updated that pesky connection string. A bad test is better than no test at all. I guess that leads on to my next topic…false positives and false negatives. But for now get testing and buy a diesel.

Ok Mr. Customer…your software is late what are you gonna do now?

2009-06-09T13:52:00.001-07:00

How do projects get late? Well as Brooks says in the seminal Mythical Man Month, “projects get late one day at a time”. That day the developers spent listening to that lousy presentation or fixing their development environments instead of delivering features, the project was getting late. That day the developers spent time chasing requirements because the product owner could not be bothered, the project was getting late. Kind of reminds me of the boiled frog metaphor.

Scrum and daily releases mitigate this risk, but the can only succeed in an enabling context.

No guts no glory…

2009-06-09T12:21:00.001-07:00

Many enterprises pay lip service to agile. They wax lyrical about agility, but the minute one practices the principles that espouse agility, they run for cover in the command and control cave. This cavern contains the usual paraphernalia of document heavy processes, bottled-necked decision throughput and the waterfall methodology. Funny really, but not surprising as some corporate types are quick to latch on to the latest buzzwords without understanding the true essence of the discipline.

As they say…”talk is cheap”. The agile acid test has got to be: ”if it talks like agile;walks like agile and smells like agile then it is agile. A form of agile duck-typing if ever there was such a thing.

The Last Mile

2009-05-23T06:16:00.001-07:00

How do you walk your software product through the wilderness of dumb users, dodgy machine builds and missing dependencies?

You want to reduce your support footprint and the dysfunction of dealing with false positives(i.e. it’s your software not my machine syndrome). If you have control of the deployment landscape then a standardised machine build and knowledgeable first line support come in handy. Otherwise online users forums and comprehensive user documentation will help share knowledge of work arounds, gotchas and technical Voodo. The last thing you want is for the development team to be constantly building patches to support all machines under the sun. Doing this robs the team of the precious time required to develop new features and enrich the product’s functionality.

Time and Tide and Software releases…

2009-05-23T05:57:00.001-07:00

My team releases software every week regardless of the political landscape or inefficiencies of other departments. Ours is a disciplined and rigorous approach, we aim to be proactive and to keep the momentum forward, never backward. After all this is agile and we embrace the iterative and incremental nature of our game. To the gentlemen from the hydro-methodology this does not make sense. Surely you should only release when the business say so and when the business are ready for your release. If we did that then we would be constrained by the same inefficiencies that plague other teams. And that aint agile. So I suppose time and tide and software releases wait for no man.

Refactoring to a fluent configuration interface

2009-05-02T04:28:00.001-07:00

The starting point was a configuration class that had the following definition:
public const decimal AdultFactor = 6;
public const decimal ChildrenFactor = 3;
public const decimal GasKwHMinEstThreshold = 4000;
:
These parameters are always used together in the context of a calculation for gas consumption. The client code uses these config values in the following fashion:

calculatedUsageKw = AdjustUsageForRadiators(calculatedUsageKw, GasConfiguration.AdultFactor,
                                            GasConfiguration.ChildrenFactor,
                                            GasConfiguration.RadiatorFactor,
                                            GasConfiguration.PersonsAtPropertyFactor);

I wanted to give the setting of the gas consumption parameters a fluid feel in the following style

public static IFluentConfiguration Create()
{
    var fluentConfig = new GasConfigurationFluent();

    fluentConfig.SetAdultFactor(6).SetChildrenFactor(3).
        SetGasKwHMinEstThreshold(4000).SetGasPointsMinThreshold(12).
        SetGasPointsToKwHConvFactor(400).SetGasWaterHeatingAmount(10).
        SetHobGasCooker(4).SetMainRoomFireOnGas(16).
        SetPersonsAtPropertyFactor(2).SetRadiatorFactor(5);

    return fluentConfig;
}

The client code accessing the fluent configuration interface was refactored to:

private decimal? AdjustUsageForRadiators(decimal? calculatedUsageKw, IFluentConfiguration fluentConfig)

The GasConfigurationFluent has setters that take the folllowing general approach:
public IFluentConfiguration SetAdultFactor(decimal? adultFactor)
{
    AdultFactor = adultFactor;
    return this;
}
And yes my setters have a return value which is in effect the current context i.e. my fluent configuration.After applying this refactoring, a few things became apparent,
1. The method chaining enforces the fact that these parameters exist in the same context and somewhat provides a cohesive view of my configuration interface. The setting of the configuration values is much more concise.
2. The cohesive nature of this configuration hints at the fact that a DSL(Domain Specific Language) could possibly be built around the calculation of gas consumption.

Wide laps

2009-05-01T09:56:00.001-07:00

If the benefits of having a wide sturdy lap were ever in question, then look no further. On the left is my company laptop firmly glued to my thigh. The other limb is supporting my XPS M1330; looking on in the background is the Inspiron 1525.

Yesterday’s scrum

2009-04-30T14:04:00.001-07:00

created at TagCrowd.com

Dinner @ the inn

2009-04-23T11:33:00.001-07:00

Yesterday was St. George's day and in celebration of the patron saint of England, I went to the local Inn where they had layed out the very best in English cuisine. This ranged from the curiously named toad in the hole to the delicious Shepherds Pie.

When Paradigms Collide- Sorting Spatial Data using the List Metaphor

2009-04-19T14:46:00.001-07:00

Given a spatial dataset that comprises physical locations that must be traversed efficiently, what’s the simplest solution that would accomplish this task?

Trouble booting from CDROM on Sunblade 100.

2009-04-19T12:20:00.001-07:00

Possible causes:

Corrupted media
Faulty CDROM drive
Non-bootable media
SunOS 5.9 limitations booting Ubuntu/Solaris 10

A lean view of scrum

2009-04-19T10:50:00.001-07:00

The sources of waste are:

Third-party delays in providing inputs required for the upcoming sprints
Stories with insufficient detail to ensure that the team proceeds without interruption
Scrum is yet to gain traction; the old way of doing things still haunts the project
Delayed feedback
Developer distractions – meetings; problems with machines; network connectivity; complex build process

NIH…

2009-04-19T10:35:00.001-07:00

When a developer inherits a codebase, there is always a case of NIH(not invented here) syndrome. I am guilty of it, but with experience and professional maturity I have become more objective in my evaluation of third-party codebases. Recently, I had the chance to work on a codebase that ticked all the boxes of what a good software application should contain. The codebase exhibited the following characteristics:

Unit testing and Mock objects
Dependency Injection
IoC Containers
MVC
N-tier architecture(and a slight hint of domain driven design)
Fluent Interfaces
NANT builds
Code-generation
Template meta-programming(generics)

These are fairly advanced concepts in software engineering and represent the software engineering zeitgeist. Unfortunately these practices will be lost on the majority of enterprise developers(unless you are working in a dedicated software shop that’s at the bleeding edge). For the typical enterprise developer, this translates into is software that is harder to maintain, extend and for which the sprint velocity is going to be dismal. The inheritors of the codebase have the additional task of understanding the domain and the engineering approach.

The question then becomes: how do you balance the need to create supple software systems whilst staying true to the software engineering discipline. Can you sacrifice the need for rigour in favour of comprehension and program understanding or vice-versa?

From XP Developer to ScrumMaster

2009-04-19T10:06:00.001-07:00

As luck(or misfortune) would have it, I have been tasked with ensuring that the team delivers the right software on time and to the product owners’ satisfaction. My previous forays in extreme programming(XP) mean that there is zero cost of adoption of this new way of working. It could be argued that scrum and XP are polymorphic. They both inherit from the same agile shrub.

It is tempting to approach the scrum master role from a developers’ perspective given that I am a developer to the core. But this misses the essence of being a scrum master. I need to balance the demands of the product owner with those of the development team whilst being aware of the latent political interests that subtly influence the project. Some of these interests have a less than positive impact on the project and are impediments that need to be reported if the project is not to grind to a halt.

Agile is new to the organisation and the idea of product owners, sprint velocity, burn-down, sprints, product backlogs, sprint backlogs is alien in certain quarters. It is not uncommon to be asked to provide an estimate on the spot without consulting the team. My advice is don’t do it and don’t over commit the team.

Mobile blogging

2009-04-18T14:57:00.001-07:00

First entry from Android G1. Mobile blogging is here...

HPC Windows Project back on track

2009-03-24T03:34:00.000-07:00

After a turbulent winter, I have now resumed work on the beowulf cluster project. All my weekends and evenings for the next quarter are already spent. I wonder how I will sell this proposition to the girlfriend. Perhaps a chilvarous pitch might do the trick..."Darling, it's a climate change calculation engine..."

The next few weeks will see me configure the head node and install the SUN GRID ENGINE tools. Then the slave nodes will be configured and the obligatory benchmarking LINPACK tools will be executed to determine the cluster performance.

We have a release candidate

2009-03-24T03:29:00.000-07:00

After weeks of tireless effort, we have a candidate release for the product I am currently working on. It's WPF all the way through. We have successfully ported a Winform application to WPF. The WPF Toolkit has proved in making this migration painless. We can use controls that we have come to know and love on the Winform UI. Datagrids, DatePickers, DateTimePickers you name it the WPF ToolKit has them. We have had to make changes to the way these controls work and this is all possible because we have the sourcecode available.

Good offshore code

2009-03-09T10:46:00.000-07:00

Is as abundant as chicken teeth.

MVVM a view gone too far?

2009-03-07T04:54:00.001-08:00

After spending a week spiking this for a tablet PC application, it quickly dawned on me that the ViewModel, the data specialised for the view is really an adapter pattern. The model is adapted to suit the expectations of the WPF UI. Adapting the model for the view means that you can essentially ignore converters.

//TODO:Add code

“But it’s auto-generated, so why does it need to be unit tested?”

2009-03-07T04:45:00.001-08:00

I recently overheard a conversation in which one developer was explaining the legacy code he had inherited to a new starter on the team. The developer described the code generators and the accompanying unit tests for the generated artifacts. Naturally the new developer asked why they needed unit tests if they were automagically making this stuff. This is a valid question and one which does beg a decent answer. I will try to answer the question based on my experience as a test-infected developer.

My response is as follows:

Auto-generation of software artifacts does not guarantee correctness. This is a garbage-in-garbage-out view of code generation. Engineers need to be sure that the changes that will inevitably occur in the code generation templates do not introduce defects. By running unit tests against the generated code you are testing that the generators works as they should and that specified inputs produce predictable outputs. The unit tests enforce the implied contract which stipulates that my code generator should always produce artifacts that do X.
Unit tests provide a reliable and repeatable way of regression testing the outputs of the generated artifacts. A change to the code templates might introduce subtle bugs. How else would you be able to detect these in the absence of automated unit testing. You are covering your own hide by having a battery of tests.
Unit tests allow you to develop with courage. You get immediate feedback when defects are introduced into the codebase (this assumes you are continuously running your tests, which you should).
The quality of the unit tests also dictates the testing experience. Several heuristics exist for developing good tests.

Working with legacy code

2009-03-06T17:31:00.000-08:00

I have recently been tasked with migrating an MVC winform application to WPF. Looking at the existing codebase, I cannot reuse any of the controllers because they are very aware of the winforms view. This is fine because the controller handles gestures from the view and should thus know the view for which it controls. So we are faced with migrating the behaviour in our winforms-centric controllers to WPF controllers.

The model code is well developed as is the accompanying repository. However the need for master/detail views in WPF means that we are going to have to alter the readonly properties in the domain objects. But do we really want to change the legacy code?

Who should manage the layout of my modular UI (WPF context)?

2009-02-21T11:25:00.001-08:00

The easy answer is : “the LayoutManager”. Does layout responsibility rest with the view or controller(ref MVC)? Whose business is it to know about the positioning of modular control xxx and whether it should be visible when an element in modular control zzz is clicked.

How do I make the best use of the MVC triad when I need to precisely manage my modular UI layout.

Do I need a view specific layout controller that is pixel and coordinate aware, perhaps a variation of the MVCC? Should the additional C reside in the xaml code behind or can I weave in the layout rules at runtime? When the scope for cross window navigation is severely restricted what pattern can I turn to?

My current problem domain brings these questions to the fore.

GUI layout tends to be orthogonal to the core concerns of application development, yet it is critical to the success and acceptability of the application. To the user, the UI is the application.

I will be prototyping various approaches to this vexatious issue in an attempt to develop a maintainable approach. At present a finite state approach in which the main actors are the view and some layout policy seems likely.

When everything is a DependencyProperty, life’s a beach…

2009-02-21T11:00:00.001-08:00

When faced with a requirement to automatically generate parts of a modular UI based on events raised by interaction with other controls, the question is normally, how do I position the generate controls in such a way that there is no conflict between my design-time view of the UI and it’s runtime view.

Combining the grid layout and use of DependencyProperty resolves this dilemma

public VanillaLayoutTemplate()
{
    InitializeComponent();
    PositionGroupControlsInFirstColumn();
}

private void PositionGroupControlsInFirstColumn()
{
    CaptureOutComeGroup.SetValue(Grid.ColumnProperty, 0);
    ProductServicesGroup.SetValue(Grid.ColumnProperty,0);            
}

The CaptureOutComeGroup and ProductServicesGroup represent a hidden control that is positioned in the first column. When the page is rendered, the 2 controls are overlapped and hidden; visibility is managed by the user providing various inputs.

WPF Toolkit and the DatePicker Control

2009-02-21T10:45:00.001-08:00

The journey started out with a hunt for a date control that would form part of the user interface. The core WPF API does not come equipped with this capability so I decided rather than lose precious time trying to build one, I would Google for it.

A download from CodePlex later, I had the binaries on the development machine and I was ready to rock. Lo and behold…there was none of the precious documentation that us developers have become fat on, but as I soon discovered integrating the control is pretty simple.

I added the new assembly reference to VS2008 and this was loaded as shown below:

The XAML for the integrated control is

<GroupBox Grid.Row="5" Header="Call back notes" FontSize="14" Margin="1,1,1,308" Visibility="Hidden" Name="CallBackNotesGroup">
    <StackPanel Grid.Row="5" Height="218" VerticalAlignment="Top">
        <TextBox Margin="2,2,2,2" Name="textBox1" TextWrapping="Wrap" MaxWidth="300" MaxHeight="120" BorderBrush="Black" Height="100" />
        <Controls:DatePicker>                    
        </Controls:DatePicker>
        <Button Grid.Row="3" Margin="2,2,2,2" Name="button1" FontSize="14" Width="163" HorizontalAlignment="Left">Arrange call back</Button>
    </StackPanel>
</GroupBox>

The rendered UI looks something like this:

The control exhibits automatic polite validation behaviour which is activated when an invalid date string is entered in the text box. That’s one layer of validation that I don’t need to worry about!

Generating the lattice from first principles

2009-02-15T16:36:00.001-08:00

In an attempt to model the binomial lattice, I will have to generate the lattice data structure and decorate it with various properties(probabilities, prices, node Id’s etc), so I thought I would approach the problem by first generating the node Ids for each time-step. So here we go:

public void GenerateLatticeNodes()
{
    var numberOfTimeSteps = 20;
    var nodes = new Hashtable();
    var nodeId = 1;
    for (var i = 0; i < numberOfTimeSteps; i++)
    {
        var numberOfNodes = i + 1;
        var nodesAtTimeStep = new int[numberOfNodes]; ;
        for (var j = 0; j < numberOfNodes; j++)
            nodesAtTimeStep[j] = nodeId++;
        nodes.Add(i,nodesAtTimeStep);
    }
    //What no test?
}

For the moment this is just a collection of identifiers to which the object richness( the notion of a node having asset price, probability of up/down movement etc) will be added.

The node numbering scheme adopted is such that the initial node is 1 and subsequent nodes are numbered incrementally in an up/down fashion.

Probability diffusion in the binomial lattice

2009-02-09T03:01:00.000-08:00

Given that we can express the diffusion of asset prices in the binomial tree as a summation of the form

The expected stock price at a given time-step can be expressed in terms of the probability, p, of an upward movement, u, of the underlying asset. Conversely the probability of a downward movement in the asset price is denoted as 1-p, where d, is the amount by which the stock moves down. Hence the expected asset price at the first time-step can be summarised as:

Why the vectors? Well I suppose there is a directional, albeit unidirectional, aspect to the diffusion. We are interested in heading towards the future and ascertaining the expected value of the asset at the expiry date.

The obvious question then becomes, “how do we generalize the findings above for a binomial tree with n time-steps?”

The probability diffusion for each time-step appears to follow the summation above. A few tests should be enough to prove whether this is actually the case.

Modelling the growth of asset prices in a binomial lattice

2009-02-06T13:58:00.001-08:00

Why would we want to do this? Well one of the key decisions when pricing call options using a binomial lattice is whether it is optimal to exercise the option or let the option lapse. We need to know whether we are in the money or not. The value of the asset price in relation to the strike price is a good indicator. If the asset is cheaper in the spot market we let the option lapse, otherwise we exercise the option, buy the asset at the strike price and sell it on the spot market making ourselves a tidy little profit :).

Starting at time t=0 and given that the volatility of the asset is d and u, the asset prices for a given time-step follow the distribution:

We then proceed to test the assertion above using NUnit 
(note the syntactic sugar which makes the testing of collections much more direct)

[Test]
public void StockPriceGrowth()
{
    var t = 2;
    var u = 1.1;
    var d = 0.9;
    var initialAssetPrice = 20;
    var stockPrices = new ArrayList();
    var expectedAssetPrices= new[] {24.2,19.8,16.2};

    for (var i = 0; i <= t; i++)
    {
        var stockPrice = Math.Round(initialAssetPrice * (Math.Pow(u, t - i) * Math.Pow(d, i)), 2);
        stockPrices.Add(stockPrice);
    }            
    Assert.That(stockPrices,Is.EquivalentTo(expectedAssetPrices));
}
And hooray we get the greenbar!

 
The stock price generation behaviour belongs to a utility class responsible for generating the asset prices
at given time-step, so we refactor the test to the following:
[Test]
public void StockPriceGrowth()
{
    var t = 2;
    var u = 1.1;
    var d = 0.9;
    var initialAssetPrice = 20;
    var expectedAssetPrices= new[] {24.2,19.8,16.2};
    var stockPrices = BinomialCalculator.StockPricesForTimeStep(t,u,d,initialAssetPrice);
    Assert.That(stockPrices,Is.EquivalentTo(expectedAssetPrices));
}

Generalising the binomial lattice for pricing European options

2009-02-05T15:49:00.001-08:00

As luck would have it, I am now building a binomial lattice pricing model that employs the concepts of risk neutral valuation, risk free rate and expected option values at a given time step. I am approaching the problem from a computational angle; I am investigating the computational efficiency and complexity of the resulting algorithm as well as the parallelizability of the code. For each node in the lattice I aim to calculate the polynomial representing the probability function. Summing these node polynomials for a given time step and applying backwardation should yield the net present value of the of the option. The scope of this investigation is limited to European call options, but with more work I should be able to extend this to price American options.

Southern Railway broken payment processing workflow

2009-02-04T11:35:00.001-08:00

Just how bad does it get? It seems the failure of the train system is really systemic. How could this happen on a live site that takes payments? It does not fill me with a lot of confidence; I guess the passwords are probably stored in clear text in the database.

This little episode transpired whilst I was trying to purchase a season ticket(annual train ticket). It happened right at the end of the workflow. The culprit: a missing include file; does this stuff ever get tested. Just because it works in the dev environment does not guarantee that it will work in the live. You would have thought that the deployment process, plus configuration management and the army of QA testers would have caught this, but it seems they never made it into the office due to the snowy conditions. Sigh.

Automated unit and functional tests would most probably have created visibility of this issue.

Integrating NHibernate with WPF

2009-01-11T12:46:00.001-08:00

A few things to watch out for when you do this integration:

Ensure that the hibernate.cfg.xml file is present.
Set the properties for the hibernate configuration file so that it is always copied to the output folder(bin/debug for debugging),

failure to do will result in the following error: “An exception occurred during configuration of persistence layer.”

Converting an XBAP to a WPF application

2009-01-11T09:36:00.000-08:00

After starting out on the XBAP route and quickly hitting the sandbox limit it was time to actually move to an application paradigm that would support file access as well as database interaction. So I decided I had to convert the XBAP to a plain old WPF application.

I added a new Window1.xaml file and reconfigured the start url in the App.xaml file to point to the newly added file.

After messing around with the property sheet for the solution, I proceeded to compile the solution and instead ended up with the following error message.

There was not an option to change the setting until I poked in the *.csproj( the project definition file) file which contained the relevant xml nodes <IsWebBootstrapper>, <HostInBrowser> and <TargetZone>. Delete these nodes and F5 the project.

<PropertyGroup>
  <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
  <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
  <SchemaVersion>2.0</SchemaVersion>
  <ProjectGuid>{DDA5DF2A-BCAF-4B2E-9A86-7E7F5F8292C7}</ProjectGuid>
  <OutputType>WinExe</OutputType>
  <AppDesignerFolder>Properties</AppDesignerFolder>
  <RootNamespace>CharityWorkSpace.Zainco.YCW.CRM</RootNamespace>
  <AssemblyName>CharityWorkSpace.YCW</AssemblyName>
  <TargetFrameworkVersion>v3.5</TargetFrameworkVersion>
  <FileAlignment>512</FileAlignment>
  <ProjectTypeGuids>{60dc8134-eba5-43b8-bcc9-bb4bc16c2548};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
  <WarningLevel>4</WarningLevel>
  <EnableSecurityDebugging>true</EnableSecurityDebugging>
  <StartAction>URL</StartAction>
  <HostInBrowser>true</HostInBrowser>
  <TargetZone>Internet</TargetZone>
  <GenerateManifests>false</GenerateManifests>
  <SignManifests>false</SignManifests>
  <ManifestKeyFile>Records Management_TemporaryKey.pfx</ManifestKeyFile>
  <ManifestCertificateThumbprint>3327D64CAF3D6C7423AD45AC8121C526DB8AB168</ManifestCertificateThumbprint>
  <IsWebBootstrapper>true</IsWebBootstrapper>

StaleStateException

2009-01-07T05:22:00.001-08:00

NHibernate.StaleStateException: Unexpected row count: 0; expected: 1

at NHibernate.AdoNet.Expectations.BasicExpectation.VerifyOutcomeNonBatched(Int32 rowCount, IDbCommand statement)
at NHibernate.AdoNet.NonBatchingBatcher.AddToBatch(IExpectation expectation)
at NHibernate.Persister.Collection.AbstractCollectionPersister.Recreate(IPersistentCollection collection, Object id, ISessionImplementor session)
at NHibernate.Action.CollectionRecreateAction.Execute()
at NHibernate.Engine.ActionQueue.Execute(IExecutable executable)
at NHibernate.Engine.ActionQueue.ExecuteActions(IList list)
at NHibernate.Engine.ActionQueue.ExecuteActions()
at NHibernate.Event.Default.AbstractFlushingEventListener.PerformExecutions(IEventSource session)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
at NHibernate.Transaction.AdoTransaction.Commit()
at Zainco.YCW.Tests.Group.AddYCWGroup() in Group.cs: line 70

This usually means that your database is in an inconsistent state(now say that quickly):

UPDATE MyTable SET Description=’xxxx’ WHERE Id=123;

If an updateable record is not found , then the exception above is thrown. Ensure that you have applied the cascade attribute on the relevant associated objects.

Joined-Subclass exception

2009-01-02T10:54:00.001-08:00

NHibernate.MappingException: 'extends' attribute is not found.

at NHibernate.Cfg.XmlHbmBinding.ClassBinder.GetSuperclass(XmlNode subnode)
at NHibernate.Cfg.XmlHbmBinding.MappingRootBinder.AddJoinedSubclasses(XmlNode parentNode)
at NHibernate.Cfg.XmlHbmBinding.MappingRootBinder.Bind(XmlNode node)
at NHibernate.Cfg.Configuration.AddValidatedDocument(NamedXmlDocument doc)

NHibernate.MappingException: Could not compile the mapping document: Zainco.YCW.Components.Mappings.Member.hbm.xml

at NHibernate.Cfg.Configuration.LogAndThrow(Exception exception)
at NHibernate.Cfg.Configuration.AddValidatedDocument(NamedXmlDocument doc)
at NHibernate.Cfg.Configuration.ProcessMappingsQueue()
at NHibernate.Cfg.Configuration.AddDocumentThroughQueue(NamedXmlDocument document)
at NHibernate.Cfg.Configuration.AddXmlReader(XmlReader hbmReader, String name)
at NHibernate.Cfg.Configuration.AddInputStream(Stream xmlInputStream, String name)
at NHibernate.Cfg.Configuration.AddResource(String path, Assembly assembly)
at NHibernate.Cfg.Configuration.AddAssembly(Assembly assembly)
at NHibernate.Cfg.Configuration.AddAssembly(String assemblyName)
at NHibernate.Cfg.Configuration.DoConfigure(IHibernateConfiguration hc)
at NHibernate.Cfg.Configuration.Configure()
at Zainco.YCW.Components.Utils.NHibernateHelper.get_Session() in NHibernateHelper.cs: line 13
at Zainco.YCW.Tests.MemberTest.AddMember() in MemberTest.cs: line 70

If you get this exception then you have probably defined your joined-subclass outside the parent class, which is probably not a good idea if you are aiming to have highly cohesive and modular mapping files. The problem is easily resolved by either adding an extends attribute to the joined-subclass element

<joined-subclass name="KeyLeaderNationalTeamTasks"  table="KeyLeaderNationalTeamTasks" extends="Task">

or nesting the joined-subclass in the class definition.

It’s the domain stupid!

2008-12-31T04:51:00.001-08:00

OR mapping ,regardless of the mechanics employed, shifts the weight of enterprise software development from thinking about persistence to actually cogitating the problem space. I am now forced to think long and hard about the best aggregation or composition approaches for each of my domain objects. And this is precisely what my customer is paying me for. As Ayende says, “persistence is a solved problem”.

Mapping Subclasses in NHibernate

2008-12-30T03:08:00.001-08:00

ContactTest.TestAddContactNotes : Failed

NHibernate.MappingException: No discriminator found for Zainco.YCW.Components.Domain.ContactNote. Discriminator is needed when 'single-table-per-hierarchy' is used and a class has subclasses

at NHibernate.Mapping.SingleTableSubclass.Validate(IMapping mapping)
at NHibernate.Cfg.Configuration.Validate()
at NHibernate.Cfg.Configuration.BuildSessionFactory()
at Zainco.YCW.Components.Utils.NHibernateHelper.get_Session() in NHibernateHelper.cs:line 13
at Zainco.YCW.Tests.ContactTest.TestAddContactNotes() in ContactTest.cs:line 94

This was due to an incorrect mapping strategy. Instead of using the table per hierarchy strategy, I resorted to using the table per subclass mapping strategy. The base table, Note, contains the elements common to my subclasses and the ContactNote maintains the association between the contact and the Note. I have a variety of Notes in the system(CommunityNotes, ParishNotes etc).

The mapping file for this strategy is:

<class name="Note" table="Note">
  <id name="Id" column="Id">
    <generator class="native"/>
  </id>
  <property name="Content"/>
  <property name="DatePublished"/>
  <joined-subclass name="ContactNote" table="ContactNote">
    <key  column="NoteId"/>
    <many-to-one class="Contact" column="ContactId" name="Contact"/>
  </joined-subclass>    
</class>

Refactoring to DetachedNameQuery

2008-12-20T10:27:00.000-08:00

The consequence of hiding the NHibernate ISession type from the client code is that I now have to rely on inline SQL which kind of makes me queasy. But this feeling is temporary because I can use a feature of NHibernate called DetachedNameQuery to achieve the following:

IRepository repository = new NHibernateRepository();
IDetachedQuery query = new DetachedNamedQuery("GetMaxTaskId");
ITask task = repository.Find<Task>(query);

GetMaxTaskId is defined in my NHibernate mapping file, Task.hbm.xml.

The entry takes the following form:

<query name ="GetMaxTaskId">
<![CDATA[
select max(task.Id) from Task task
]]>
</query>

This really does feel good, especially when you type like I do

2008-12-20T09:13:00.000-08:00

ITask task = new Task { Description = "Coding for fun", DueDate = DateTime.Now, IsCompleted = false, Categories = new HashedSet() };

contrast with the old way of doing things

ITask task = TaskFactory.Instance;
task.Description = "Yak Shaving";
task.IsCompleted = false;
.
.

It's the small things like this that make for a more productive coding experience. The intent is more direct and one quickly gets to the destination in a DRY way. Nice!

NHibernate.LazyInitializationException: failed to lazily initialize a collection, no session or session was closed

2008-12-20T08:47:00.000-08:00

I have implemented a repository that essentially wraps up and centralises my calls to the NHibernate API. This facade is responsible for creating and disposing the NHibernate session.

Clients do not directly interact with the session.

So I use the IDetachedQuery interface to hold my queries like soz:

IDetachedQuery query = new DetachedQuery("...");

ITask task = repository.Find <Task>(query);

My task has a collection of categories which I am attempting to clear, but the session with which this collection is associated has already been disposed, so looks like I am going to have to force the collection to initialise before I return from the find method. The other option is to eagerly load the stuff i.e. set lazy loading to false.

As in Hibernate non-lazy loading is the default for loading collections in NHibernate. The solution to this problem was a case of setting the lazy attribute to false like this:

<set name="Categories" table="TaskCategory" lazy="false" cascade="all">

NHibernate can also be forced to initialize the collection by calling NHibernateUtil.Initialize(myObject) before the session is disposed.

Perhaps NHibernate Mapping attributes are poisonous

2008-12-18T17:33:00.001-08:00

I have spent the best part of a working day experimenting with using NHibernate atrributes after starting off with .hbm.xml mapping files.

The result is a domain model that's very NHibernate aware, I cannot reuse the model without there being a whiff of NHibernate. This being the case, I have decided that xml mapping files are probably the way forward. I am aware of other mapping approaches, but perhaps, the verbosity and loosely coupled approach of mapping files has its advantages.

Yes attributes and mapping approaches use a declarative style, but one can not help feeling that embedding persitence crosscutting concerns in domain model could cause grief further down. Mapping files, you have a new friend.

An NHibernate Helper for working with the NHibernate.Mapping.Attributes API

2008-12-09T10:27:00.000-08:00

Authoring, maintaining and managing NHibernate mapping files, hbm.xml, can quickly turn into a full-time job on large projects. One way of effectively accomplishing this vital, but burdensome task is to use the NHibernate contrib, NHibernate.Mapping.Attributes.

This will requires configuration information to be placed in an app.config file as documented in the NHibernate documentation. One is then required to have a single well known location from where to obtain an instance of the ISession(think PersistenceManager) type. The code below shows how to configure the serializer and return an ISession:


using System.IO;
using System.Reflection;
using NHibernate;
using NHibernate.Cfg;
using NHibernate.Mapping.Attributes;

namespace Zainco.YCW.Components.Utils
{
    public sealed class NHibernateHelper
    {
        private static ISession _session;
        public static ISession Session
        {
            get
            {
                if (_session == null)
                    using (MemoryStream stream = new MemoryStream())
                    {
                        HbmSerializer.Default.Validate = true;
                        HbmSerializer.Default.HbmNamespace = "Zainco.YCW.Components.Domain";
                        HbmSerializer.Default.HbmAssembly = "Zainco.YCW.Components";
                        HbmSerializer.Default.Serialize(stream, Assembly.GetExecutingAssembly());

                        stream.Position = 0;

                        Configuration cfg = new Configuration();
                        cfg.Configure();
                        cfg.AddInputStream(stream);
                        _session = cfg.Configure().BuildSessionFactory().OpenSession();
                        
                    }

                return _session;
            }
        }
    }
}

System.Data.SqlClient.SqlException: Invalid object name 'hibernate_unique_key' when using SQL Server 2005

2008-12-09T09:28:00.000-08:00

This problem occurs when attempting to persist an entity

the actual code that causes this is:

session.Save(contact);

The Id attribute on the contact entity is configured as:

[Id(Name = "Id", Type = "int")]
[Generator(Class = "hilo")]
public virtual int Id



ContactTest.TestAddContact : FailedSystem.Data.SqlClient.SqlException: Invalid object name 'hibernate_unique_key'.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader()
at NHibernate.Id.TableGenerator.Generate(ISessionImplementor session, Object obj)
at NHibernate.Id.TableHiLoGenerator.Generate(ISessionImplementor session, Object obj)
at NHibernate.Event.Default.AbstractSaveEventListener.SaveWithGeneratedId(Object entity, String entityName, Object anything, IEventSource source, Boolean requiresImmediateIdAccess)
at NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.SaveWithGeneratedOrRequestedId(SaveOrUpdateEvent event)
at NHibernate.Event.Default.DefaultSaveEventListener.SaveWithGeneratedOrRequestedId(SaveOrUpdateEvent event)
at NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.EntityIsTransient(SaveOrUpdateEvent event)
at NHibernate.Event.Default.DefaultSaveEventListener.PerformSaveOrUpdate(SaveOrUpdateEvent event)
at NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.OnSaveOrUpdate(SaveOrUpdateEvent event)
at NHibernate.Impl.SessionImpl.FireSave(SaveOrUpdateEvent event)
at NHibernate.Impl.SessionImpl.Save(Object obj)
at Tests.ContactTest.TestAddContact() in ContactTest.cs:line 20

The solution is to use the native option, this picks identity, sequence, hilo depending on the capabilities of the underlying RDBMS.

[Id(Name = "Id", Type = "int")]
[Generator(Class = "native")]
public virtual int Id
{

Permission error when trying to open a new database connection from an XBAP

2008-12-04T23:46:00.000-08:00

If you get the SecurityException below whilst developing an XBAP application

{"Request for the permission of type 'System.Data.SqlClient.SqlClientPermission, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed."} System.Exception {System.Security.SecurityException}

This is because the default security mode for an XBAP is partial trust. The XBAP is sandboxed and has restricted permissions to the file system, database etc. The .NET Framework uses code access security to limit what the XBAP can do. This is kind of consistent with the Flash, Java applets paradigm that XBAP tries to emulate.

It is tempting to assign the application trust via the following mechanism:

In VS2008, do the following:
Double click the VS projects' Properties node containing the XBAP-> Select Properties->Choose the security tab-> Select the option which says "This is a full trust application"->

However this Full Trust permissioning has potential drawbacks that require unpalatable workarounds.

The side effect of the change is that me XBAP can not now be run from a virtual directory or website.

This leaves me to wonder. Is what I need an XBAP, SilverLight application, a standalone XAML application or God forbid an ASP.NET app?

I still want the rich internet application experience and I still want to have a centralised web-like configuration management model. The last thing I want is to be touching each clients' machine to tweak a setting here and a setting there. Because this approach inevitably ends in tears.

I also do not want to implement orthogonal constructs that check for permissions and end up littering the codebase with try/catch blocks that test for permissions.

So in the end after careful consideration of my problem context, SilverLight is the way to go.

A unit testing framework in Turbo Pascal

2008-12-04T22:22:00.000-08:00

Why Turbo Pascal? Because it reminds me of the joy that I went through trying to learn/reason about software construction and partly because of the turbo button on my first computer. I had it permanently on as if it that would make the world go faster. Now we have multi-core machines, hyperthreading, desktop Beowulfs and grids. Hey but I digress.

The main point of this exercise is to implement an xUnit testing framework using Pascal. This follows on from a segment in Kent Becks' book, Test Driven Development, in which Mr. Beck implements an xUnit framework in python.

TODO add code here.

Type registry and the Factory Method

2008-12-04T15:36:00.000-08:00

When implementing the Gofian Factory Method creational pattern it is normally the case that the client requiring the instance needs to tell the object factory what kind of object it needs to make. The internals of the factory method that decide what object to return usually resembles the following


public class Creator
{
:
public static IVehicle Create(VehicleType vehicleType)
{
  IVehicle vehicle = new Chisasa();
  if(VehicleType.Sedan)
     vehicle = new Sedan();
  if(VehicleType.FourByFour)
     vehicle = new FourByFour();

  return vehicle;
}
}

This decision structure is easily refactored to an in-memory type registry which uses the lookup metaphor as demonstrated below:


private IDictionary<VehicleType,IVehicle> Vehicles
{
get
{
  IDictionary<VehicleType,IVehicle> vehicles = new Dictionary<VehicleType,IVehicle>();
  vehicles.Add(Vehicle.Sedan, new Sedan());
  vehicles.Add(Vehicle.Mazembe, new Mazembe());
  vehicles.Add(Vehicle.Chisasa, new Chisasa());
  vehicles.Add(Vehicle.Chimbayambaya, new Chimbayambaya());
  return vehicles;
}
}

public static IVehicle Find(VehicleType vehicleType)
{
 return Vehicles[vehicleType];
}

If you are observant, you will notice that this changes the semantics of the implementation. We have now moved from creating objects to finding objects in a registry. Sound familiar? This emergent design shows us that the Factory Method is really a respository pattern in disguise. Refactoring a GOF pattern has teleported us to one of the basic tenets of Domain Driven Design, the Repository Pattern.

Some considerations to take into account would be deciding the best collection to use
to reduce the cost of lookup. It could also be possible to externalise the registry depending on the nature of the problem space.

The characteristics of a good software service

2008-11-29T05:17:00.000-08:00

From a software clients' perspective, a good software service will exhibit the following characteristics:
1. Simple to use. You ask it for stuff and it gives you back precisely what you want. If there are any failures, these are reported. The service should fail fast. No slow insidious failures should be allowed.

2. The chunkiness or chattiness of the API should take into account the useage context. The cost of remote invocation should be factored into the design. In a distributed enviroment it might be worth measuring the cost of providing a chunky API vs a chatty interface.

3. The service should expose an interface for metadata and other out-of-band information.

4. The service should be interoperable - dont tie the service to platform specific return types unless the requirement explicitly states so or there is a compelling performance reason to do so. This assumes that you have measured the benefits of doing so.

From an implementation perspective it is desirable that the service exhibit the following attributes:
1. Statelessness - an invocation of the service at time t + 1 should be independent of the invocation at time t. The services' invocation history is not important. The "now" is more important than what has gone before.

2. Operations should be defined interms of concepts in the domain. Parameters and return values should be defined interms of domain objects. Beware of primitive obsession, i.e passing strings, int, floats etc when you should really be returning well encapsulated objects or passing parameter objects.

Why the Repository Pattern Sir?

2008-11-29T04:58:00.000-08:00

When applying domain driven design, the repository pattern yields the following benefits:
* Represents all objects of a certain type as a conceptual set
* Provides access to roots of aggregates
* Can return summary information
* Provides mechanism for object add/delete
* Encapsualates DB access technology and strategy
* Answers the question, "How do I get a handle to an object" - done via object traversal, direct query execution to find object
* Clients of the repository interface use query based methods to get objects
* Repository is a first class client to object factories. The repository interface can encapsulate factories.
* Simplifies mock testing

Interface based repository implementations promote substitutability which is a naturally results in highly configurable and loosely coupled software architectures as in Liskov Substitution.

When to use specific WPF layouts

2008-11-29T03:03:00.000-08:00

1. Grid- Does what it says on the tin. Provides row/column layout. Most versatile. Layout of choice for WPF window. Sizing supported via 3 strategies:
a) Absolute size - allows exact sizing via device-independent units. Inflexible and does not support changing content size, changing container size or localization.

b) Proportional size - Space divided between a group of rows and columns

<ColumnDefinition Width="*"></ColumnDefinition>

This syntax is very similar to frames

c) Automatic size - A versatile sizing mode; Row/Column allocated exact space it needs and no more.

<ColumnDefinition Width="Auto"></ColumnDefinition>

Mixing Grid Sizing Strategies
When mixing proportional sizing with other sizing modes, the proportionally sized rows or columns get the remaining space.

<strong>Assigning weights</strong>
<RowDefinition Height="2*"></RowDefinition>
<RowDefinition Height="*"></RowDefinition>

Means make the height of the row above twice the height of the row below

2. Canvas
Allows the positioning of elements and controls using precise coordinates. This is ideal for a drawing application, but a sub-optimal solution for rich dynamic GUIs. The layout is fairly lightweight because it does not include complex layout for position and sizing controls and elements. It is essentially the WYSIWYG of the WPF layout world.

Example of a canvas layout:
<Canvas>
<TextBox Canvas.Left="100" Canvas.Top="50"> Is positioned at (100,50)</TextBox>
</Canvas>

3. InkCanvas - the main aim of this layout is for stylus input. Think tablet PCs and mouse interactivity. Users can annotate content using strokes.

4. UniformGrid - a highly constrained and inflexible layout system. It does not support predefined columns and rows. The Rows and Columns are defined as attributes on the UniformGrid element. Cells are allocated the same size. Elements are rendered in the grid based on the order in which they are defined.

<UniformGrid Rows="2" Columns="2">
<Button>1</Button>
<Button>2</Button>
<Button>3</Button>
<Button>4</Button>
<UniformGrid>

This layout could be useful in a board game for example Nsolo.

Grouping RadioButtons using the StackPanel element

2008-11-26T12:32:00.000-08:00

The problem was how to group a number of radio buttons in a XAML UI. The solution was:


<StackPanel Name="magazineList" Grid.Column="1" Grid.Row="0" Grid.ColumnSpan=">

<RadioButton>By email</RadioButton>

<RadioButton>By mail</RadioButton>

<RadioButton>No magazine</RadioButton>

</StackPanel>





An alternative approach would be to use a named group descriptor.

The dark art of Netra t1 105, x86 and custom RS232/RJ45 interconnects

2008-11-26T11:07:00.000-08:00

Connecting the Netra 1s to a desktop x86 machine running window XP is proving to be a hard problem. The connection is via a custom built RS232/RJ45 female cable. This connects to the Netra 1's Serial A port. The sun server does not have any OS installed and no media drive, which leaves the null modem option as the only remote management solution. Constrained budgets mean that no extra accessories can be bought to fix the issue.

Alternative approaches are:
1. Re-purpose the RJ45/USB Connector to provide the linkage between the two devices.
2. Ask for more cash to buy SUN compatible hardware which is compatible with the problem domain.
3. Scrap the project - easy to do but not a realistic option.

IoC in ASP.NET MVC

2008-11-15T20:53:00.000-08:00

We used the repository pattern(DomainDrivenDesign) to implement an injectable repository type in the peristence controllers in our ASP.NET MVC application.



HandleError(View = "AddItemError",ExceptionType=typeof(CreateItemException))]
public class PersistenceController:Controller
{
  private readonly IPersistenceManager _persistenceManager;

  public PersistenceController(IPersistenceManager persistenceManager)
  {
     _persistenceManager = persistenceManager;
  }

   public PersistenceController():this(PersistenceFactory.CreateInstance())
   {
   }

The PersistenceFactory was introduced to decouple the concrete types created from the actual client using the type.

Chain of responsibility

2008-11-15T20:37:00.000-08:00

This chain of resposibility is from a system that handles a variety of reporting requests. The inheriting handlers are Russian dolled(each handler knows its successor).


namespace Components.Reports.QueryHandlers
{
    public abstract class ReportQueryHandler
    {
        protected ReportQueryHandler _reportHandlerSuccessor;
        protected ReportType _reportRequest;

        public ReportQueryHandler(ReportType reportType)
        {
            _reportRequest = reportType;
        }

        protected string LoadQuery(string queryFilePath)
        {
            if (!string.IsNullOrEmpty(queryFilePath))
                return File.ReadAllText(queryFilePath);
            else
                throw new ArgumentNullException(queryFilePath);
        }

        public virtual void HandleReportRequest()
        {
            if (_reportHandlerSuccessor != null)
                _reportHandlerSuccessor.HandleReportRequest();
        }
    }
}

Business intelligence in the face of object databases

2008-11-15T20:16:00.001-08:00

TODO

In search of dynamic cohesion via extension methods

2008-11-15T20:05:00.000-08:00

Extension methods are a new language construct that allow the addition of new method definitions to an existing type.
One particular scenario where they have proved useful is in extending the HtmlHelper type that allows rendering of controls in ASP.NET MVC. This encourages a belated type of cohesion, i.e., all concerns to do with control rendering in the view must reside in the HtmlHelper.


public static string CommandBuilder(this HtmlHelper helper, string controller, string action, string commandName, int id)
{
  return string.Format("<a href='/{0}/{1}/{2}'>{3}</a>", controller, action, id, commandName);
}

However one wonders whether the same could not be achieved through normal type inheritance. If there are breaking changes in the implementation from Microsoft, then be prepared to break.

Object Persistence without the impedance mismatch

2008-11-15T19:53:00.000-08:00

As part of the continued journey in exploring the corners of software development, I decided to implement a new IDb4ObjectsRepository to handle persistence via an OODBMS. After an hour of hacking, I had an ASP.NET MVC application conversing with an object database.
There were no schemas, mapping layers, sql queries or setup scripts to fiddle with.

The only problem I seemed to encounter was that of binding my object collections to drop down lists that were expecting an id for each bound object. Db4objects does not have great support for generating sequences numbers for objects being persisted.

There is an internal representation of the object id, but this requires groping into the guts of the api to retrieve it. Perhaps what's required is some interfaces e.g. ISequenceGenerator which can be implemented by entity objects requiring an id. Value objects by their very definition would not need to implement this interface.


private void AddItem(T entityToStore) where T:IEntity
{
  using(IObjectContainer db = Db4oFactory.OpenFile(Db4oService.FilePath))
  {
     db.Set(entityToStore);
   }
}

The sheer ease with which I was able to integrate Db4Objects left me a bit uneasy. Can object persistence be so easy? Why are object databases not prevalent?