Jonathan Crossland's Talkware

I have to build Forms. Web Forms, digital versions of paper forms. You know the kind. You get probably hate filling them in.
Currently there are a few forms, but expectations are, that many, many more forms are to follow. Forms for Edit, Forms with Permissions (part of forms visible to some, editable by others), forms saved as pdf, as html. Each form can expect largely varying workflows. Data from forms to varying database schemas. A complete nightmare to implement generically, but there is a need for a lot of it to sit on top of something familiar and useful across domains and clients.
Some of the goals include:


- easier to maintain, add forms etc, than previous implementations.
- easier to edit and deploy to different sites.
- customizable workflow
- customizable processing.
- Notifications, manual override of process, some manual flow points.

This is kind of like ruby on rails meets Windows Workflow foundation. But alas that wont work out very well, as a combined technology and WCF is overkill in some areas, and might be worth doing on another level. However this is a .NET team, with no WCF experience. Would a quick Activity based object model not suffice?

I have been sitting with this for a few weeks now, and I have 90% of my plan of attack. However, I have started developing already. You guessed it, I have already built a small Framework of my own that will suit this development.
If you have bright ideas, let me know.

.NET, c#, Frameworks webforms, asp.net, framework, design

Although .NET Reflecton offers a decent look at the Types you build, it is pretty generic in its offering. For example, it does not yet provide us with capabilities to control and secure how Reflection can inspect our Types. If you do not want a Type to be Instantiated via Reflection, or if you would like to restrict certain Types from being visible at all. Further there is no idea for extension or change. One cannot simply plug into Reflection and override behaviour.

High abstractions and higher flexibility

Many more wrappers are needed, at a higher level, which would sit on top of .NET Reflection. For example, architectural dependencies. Agile mechanisms, if code had enough metadata attached, one could alter the code via reflection/refactoring in a much more dynamic way. Consider the following two web service methods in two different companies.

  public class PersonService
    {
        public void AddPerson(string lastname, string firstname, 
            string birthdate)
        {
        }
    }

    public class PersonWebService
    {
        public void AddNew(string firstname, string surname)
        {
        }
    }

You can see that they are completely different contracts. Interface and contract speaking they are like apples and sand. However, to your mapping, calculating, schema, metadata, tagging and understanding brain, its the same thing.

What we need, is to remove ourselves from Contracts to something more like guidelines.

  public class PersonService
    {
        [Operation(Schema:Person, "Add")]
        public void AddPerson([schema(LastName)]string lastname, 
             [schema(FirstName)]string firstname, 
             [schema(DateOfBirth)] string birthdate)
        {
        }
    }

    public class PersonWebService
    {
        [Operation(Schema:Person, "Add")]
        public void AddNew([schema(FirstName)]string firstname, 
             [schema(LastName)]string surname)
        {
        }
    }

.NET, c#, Code Generation, Design, Design Patterns, Development Tools design patterns, frameworks, reflection, metadata

Not sure if there is a Design Pattern relating to this kind of need in another pattern catalogue, but it seemed simple enough to reverse the GOF composite. A class with a collection of leaves, that iterate, recurse its way up to the root. This needs work in terms of combining or actually making a variation on the composite. I think that this is perhaps only a .1 degree of separation.

Problem

You have a list of Items, that may or may not be related. There may be 1 or more trees. You have a bag of unsorted or related objects as a signle array. You need to maintain a complete list of leaves, but also need the relationship/heirarchy to become apparent. You want to express the heirarchy in a Composite Pattern, but it would exclude certain leaves has it has no part in the particular root that the composite gets built on.

Context

You need a collection of leaves, from across the composite tree. You need to have access to the parent and/or recurse to the root.

Types are stripped from its branches.

 

Structure

A class that works with a collection of leaf classes, that each point to it's parent. In effect a composite in reverse. Keep all the leaf objects in a collection and build the heirarchy from the bottom up.




One of the main reasons behind this is multiple paths to a set endpoint. The root becomes the set endpoint where by we can plot a course from a leaf. The Leaf though is where we start from and full knowledge of all leaves should be known at the outset. 

By adding a Parent on the Composite (GoF), you can achieve a similar result, but you will not have a flat list of leaves.

.NET, c#, Design, Design Patterns design patterns, composite, reverse composite

Run VS.NET as Administrator on Vista.

Export your settings using Tools | Import and Export Settings, don't let another windows reinstall take it all away from you.

Get Regionerate
- but make sure you use the "Remove All regions" command, before trying to apply others. It wont replace your existing regions.

Get Studio Tools, for File manager and the tear off editor.

Download Dev /efor - not sure about the name, but it works nicely.

Snippet Editor, useful, for changing the way the existing snippets work, as well as new ones.

Carl J has a large list of add-ins.

.NET, Development Tools, DOTNET, Visual Studio

It is amazing how these sorts of tools divide opinion.
I usually blog about technology, especially framework oriented things, but I have been thinking a lot about business again. How can it work for business.

For example, a company I have here in my midst, Oxford Creative, arose from building sites for the car manufacturer Seat. (They are searching for clients, so get on board quickly, as they are a small but effective team.)

I have been talking about twitter and many of my friends do not see the point.

But twitter, is less evasive, more rapid, more dynamic than any other social media tool.
I do not get hounded by it. I switch it off, its off. I switch it on, I have access to some great minds.

I found this How To Use Twitter Effectively for Business and Advocacy, which says a lot about its power, amongst many other posts, which have been floating around twittersphere.

I use it, because I can meet some interesting people, like @unclebobmartin, and @kentbeck, @bigballofmudd (Brian Foote). I like twitter because its less intrusive than phones, emails and facebook, but its powerful enough to communicate.

Other twitter, blogging, social media

Eventually Interfaces will be replaced with magnets!

Design Patterns, Frameworks frameworks, object-oriented, design patterns

In The Every Project Framework - Part 1, I spoke of partitioning, and abstracting structure and infrastructure. Now Let's talk about how we do that.

Over the last decade, the most prolific and available Architectural Pattern introduced and adopted by business developers is the Layers pattern. Call it by another name, but the concept is around everywhere. Most enterprise systems have open or closed Layers. They have many names, but Data Access Layer, Business Layer and so on, are often used. This a good thing towards raising the level of abstraction, increasing the infrastructure and providing more structure. However, having layers, does not automatically mean your application is on a good footing.

Raising the bar

If you are very skeptical, read the 'Why' section below first.

What do you think of your objects? How do you think of them? Are they transportation, containers, db table wrappers or other? Majority of business applications I see, have objects as a 1 to 1 mapping to a database table and it effectively is a container for a row of data. Some then wrap it into a collection of these rows and you have a table. Is that really what you need? is that efficient and good practice? Does it raise the bar high enough?

The first thing to say is that wrapping your table into an object is not entirely bad, as it is a level of abstraction away from the table.
However, if its at the highest level of abstraction you go to, then you will still have miles of code that you need to place elsewhere. Code such as Context, Security, Permissions, work flow and so on. If you manage all of your Types as a 1 to 1 mapping to the database tables, are you not merely writing code against the database?

If its UI (form) accessing and using -> the DataType Person, then your level of abstraction to the data is almost zero. You have abstracted the concept of a database table and replaced it with a more crude version, which you are populating.

Then the relationships and structure between all these singular Types will slowly come together within your UI, behind your button, in some method of some obscure class. Instead, the singular classes should be a little more structured and the bar should be raised higher, providing your UI with more closer, ready and easier to use objects.

To repeat in more a programmatic way, consider a table named Person.

If you have your UI --> Types --> Table

effectively you have abstracted a Row and Column concept away from the UI and replaced it with a PersonType and a PersonTypeCollection.
That means the UI, must understand how to deal with these Types.

To raise the bar, you would have more abstractions between UI and database
perhaps more like so

UI --> PersonCompositeType --> transaction(PersonDataType1/PersonDataType2) --> Table1/Table2

So PersonCompositeType is at much high level.

Please note:I am not saying that your architecture must look exactly like this, what I am suggesting is that more levels of abstraction can be achieved between the UI and your 1 to 1 mapped Type, that is so commonly found.

Why?

Have you ever had the following in your last or current project?

The UI is a lot more work than writing the "engine" or "business layers"!
Well of course it is, because the level of abstraction for the business layer is so low.

Your project starts quickly, progress seems to be made at speed, and then it slows right, right down....
It would, because, laying out database tables, getting your SQL and business objects in a 1 to 1 mapping is easy. Tying these low level abstraction together into a working application is harder.

Are you using a workflow engine? Are you using a business rules engine? Do you have isolated, stateless services? Do you have wrappers, commands, actions?
If not, then all the workflow, business rules, services are tucked away, most likely in your user interface. If you don't think you need these things, your large projects are most likely failing, running late and/or getting harder and harder to maintain.

Closing thoughts

If you think in "the ways of frameworks". Not as a framework that will be released to the world. Not as a time consuming lets focus on the framework, but rather build it, partition it and expand it within the project, your application will benefit from having more structure and more infrastructure.

Even having one of those is vastly beneficial. That is why a company with their own libraries (infrastructure), can get an application up quicker, even if its not a framework. One or both infrastructure and structure is vital for reuse and time to market. Yet, there is very little training going on at university, or courses in this area.

We should be training developers from their earliest years in software to think about reuse, to understand patterns, and putting rudimentary frameworks together.

Don't let the fact, that there are no perfect frameworks out there, lure you into thinking frameworks are bad. They are just diamonds in the rough. Most importantly, the "good" aspects of frameworks can be used within your application to high rewards. But how do you do it? -- stay tuned...

Design, Design Patterns, Frameworks, Methodology frameworks, design patterns, methodology

In The Every Project Framework - Introduction, I spoke of structure and infrastructure, lets now move a little deeper.
[more]

Structure is just like Pronutro

When you create an Application, you are immediately adding complexity and dependencies. How well you manage these is how long your application stays 'clean' or 'partitioned', but nevertheless your Application is getting structure with each method calling another, with each class depending on another.

If you don't think about a class, its name, where it should reside, or if you choose incorrectly it affects everything to come. You may not be aware of it entirely, but every decision has a consequence. Perhaps in the future we will have tools to help us deal with 10 or more steps of future planning, so we can verify our decisions a little better, but for now, we only rely on experience and intuition. (Like the movie Next with Nicholas Cage)

btw. Pronutro is a black hole for milk!

We have all heard of 'spaghetti code', but code does not need to be that bad, before it can start becoming a strain to continue with. Just one configuration file setting in 1 tiny config file, can cause masses of damage if its not there or if its incorrect. Yet for the majority of the time, developers put things into configs and rely on the fact that its there.

Convention over Configuration should be adhered to first as a rule.

Partitions
One of the most important aspects for agility within any structure is partitioning. Dividing lines between x and y. Effectively its all about low coupling. Read Grasp patterns regarding Cohesion and Coupling. But also how clean are the partitions? A clean break is when indirectly what you rely on works even if its not available. For example, I used TidyHtml to neaten some html before parsing it. If the TidyATL.dll is not registered in the registry, do I die a painful death or do i simply use the html and proceed to the next step without tidy? How you design and implement code with that sort of thought is what makes it a clean partition. Unfortunately at the extreme ends of things, our tools make this tremendously time consuming to get right for all scenarios. It is easy for example using Tidy, but a lot harder for most critical aspects of an application.

Giving Structure is a fundamental part, but below it on the "layers diagram", is the infrastructure.

Raising the Infrastructure

Today it is pretty simple to use your current modern language to create a simple calculator. the level of abstraction is almost perfectly aligned to the requirements of the small Application. For example, the textbox in .NET is perfectly sufficient, and the Types float, decimal, double, int etc are all at the 'right level' for a calculator.
However its not at the right level for a Stock Management Application or other more complex business software. This is where we need to raise our infrastructure to a more suitable level.
And most of us do, we create Loggers, Data layers and so on, to aid us in reaching the Application from a higher vantage point. But are you properly aware of what is structure and what is infrastructure? And are you raising the level of abstraction high enough?

Next post, I will discuss these levels of abstraction, Architectural Design Patterns such as Layer, from the strict vantage point of structure and infrastructure.

.NET, Design, Design Patterns, Frameworks, Methodology frameworks, architecture, design, design patterns, methodology

When creating a large project, it is sometimes extremely hard to find names for certain things. So I have a small list. I mix and match ActivityItem, PropertyItem, ElementFactory and so on. Helps with those generic classes you cant find names for!
you probably already using most of them, if not all ..... (got more for me?)
[more]

Single Item Activity Action Command Item Element Base Unit Node Property Column Row Value	Arrays of Items Array Composite Collection List Set
Contextual Info Context State Status Format Mode Level Type Scope	Controllers/Managers/Services Runtime Environment Executor Handler Engine System Controller Provider Publisher Designer Manager Factory Builder Converter Inspector Service Utility Helper Message Process Thread Reader Writer Serializer Action Command
Extension Adaptor Extension Facade Proxy Wrapper Dynamic Static
Configuration Attribute Configuration Setting	User Interface Style View Component Control Editor Widget

Design, Design Patterns naming conventions, class names

Do you think of a Framework as a large 'hole, to be avoided'? Are they too complicated to develop within your project, for everyday development?
I take the view, that a Framework can very easily be part of every project you build, and with great benefit. You just have to understand how.

A Framework has two very important characteristics to give to Applications. A Framework provides structure and infrastructure.

[more]

Structure

An application without structure is just text. An Application without structure contains no discernible patterns. If your Application contains no patterns, it becomes complicated. If you have structure and patterns, you have organization, categorization and commonality. An Application must have structure and I know that every single Application ever written has a structure. Every system ever built has patterns. Even the smallest console driven application will have a set way of dealing with arguments, and other code structure.

As developers we are inclined to think in patterns, that is why Design Patterns are familiar and majority of developers can agree that whether they use a Pattern from a book or their very own Design Patterns, a Pattern is a valid helpful tool.

Infrastructure

An Application that does not have infrastructure, is voltage. All Applications sit on levels of abstraction, whether its the assembly language, a runtime, programming libraries and/API, all Applications rely on infrastructure. The infrastructure of a given Application differs between projects, because as developers we can develop our own levels of abstraction, or rely only on those that are provided, or mostly somewhere in between. Even the smallest application will have some sort of API, or common shared library. We rely on the infrastructure provided at the bottom and the layers of abstraction all the way to our user interface.

Structure and Infrastructure are at the heart of Frameworks. Yes, a Framework can be other things as well, but at its core, a Framework offers structure and infrastructure to an Application.

Within all Applications...

there lies a Framework.

The developers may not have consciously created it, nor was it perhaps designed, but its there. It may be difficult to see, dirty, mixed up within all your code, but it is there!. The trick is to identify it and evolve it as you develop your Application. When you can separate the Application from the Framework (even if the framework is small), means you have separated along the two large dividing lines of change that will occur in your Application. Even if your Framework is not perfect and cannot be used outside of the current project, having it separate aids in the agility and life-time of the Application.

It is a craft, do not expect to build reusable complete Frameworks the first time, but as you build and think in the language of Frameworks, your skills will improve and it will become easier and easier.

"a class is to object, as a Framework is to Application"

Design, Design Patterns, Frameworks, Methodology, Refactoring frameworks

To conclude the previous two posts on using composition for refactoring complex classes

Refactoring complex classes using Composition Part 1
and Refactoring complex classes using Composition Part 2

Why should complex classes be refactored? To make code smaller and more manageable. To separate code that changes at different rates, and which are not concerned with the same things.

Composition vs Inheritance? Composition = "has a" relationship, Inheritance "is a" relationship. Inheritance quickly makes you feel like you create Object-Oriented code, but its one of those devices that should be used for a real reason, not just off the cuff.

Reading
- Some background from C2 Wiki - Composition Instead Of Inheritance
- A good principle from Evolve aggregations from inheritance heirarchies - Brian Foote and William F. Opdyke
- Look at Adaptor, Facade, Proxy from Design Patterns (GoF) and also the Stategy, which is mentioned here by Erich Gamma A Conversation with Erich Gamma, Part III
- Refactoring Catalog - Martin Fowler
- Refactoring posts - Martin Fowler

.NET, c#, Design, Design Patterns code, refactoring, oo, composition, inheritance

Please Note: Please make sure you have proper unit test coverage on your classes, as you refactor.

From Part 1, I showed you how I initially split a large class into a few more, splitting the complexity of having the code in one place. However, you will still find that for a really complex class, there will still be quite a lot to do. Here are a few more composition based things to consider for continuing the refactoring process.

[more]

Extract surrounding Exceptions

You will have to decide where you do this carefully, according to your own code, but I try to pull out the larger surrounding Try/Catch blocks into the wrapper, like Start method below. I now can remove the code from RuntimeExecutor for exceptions on as many methods as possible.

There may well be places you do not want to do this, but if your RuntimeExecutor class is private and only used by this wrapper, you should be able to remove most Exception handling in the wrapped class.
However, if the class is complex in this area, you may need an Exception Handling strategy instead. If your wrapped class, throws a complex amount of Exceptions in weird and wonderful areas within the code, you will have to think about it harder, but for most classes you can get away with extracting it.

//abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        public void Start()
        {
            try
            {
                _Executor.Start();
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }

        public RuntimeException LastException
        {
            get
            {
                return _Exception;
            }
            protected set
            {
                _Exception = value;
            }
        }
    }

Splitting by lifespan and age

Think of your class in three main stages, Constructing, Running, Deconstructing and refactor based on it. For example, move Initialization code from RuntimeExecutor to Runtime. My class has an Initialize method and some code in its constructor. I now refactor this into the wrapper Runtime. I also move initilization checks to the wrapper.

    //abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        public void Start()
        {
            try
            {
                if (_IsInitialized)
                    _Executor.Start();
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }

        private void Initialize()
        {
            try
            {
                //Initialization code, sets RuntimeState (see Part 1)
                _IsInitialized = true;
            }
            catch (Exception ex)
            {
                LastException = new RuntimeException("Execute Failed. See inner exception for details.", ex);
                Status = RuntimeStatus.Errored;
            }
        }
      
    }

I then do the same with any deconstruction code, so that they too exist in the Runtime class, and extracted from the RuntimeExecutor

Conclusion

The Utility class we created to move those more stateless methods into, can be expanded a little more, by moving even more methods from the main class into it. If you need to, you can even pass the RuntimeState instance via the constructor through to the class, so it can now have state. This will allow you to move even more methods into the Utility class. Remember this class is private and owned by the RuntimeExecutor. If some of these methods are more generic than that, you need to move it out into other classes.

We created four classes from one bulky one. Here is an example in code, of what we did.

 /// 

    /// We refactored UP, and created a Wrapper
    /// We moved wrapper methods, exceptions to this class
    /// 
    public class MyTypeWrapper
    {
        MyType _MyType;
        MyTypeState _MyState;

        public MyTypeWrapper()
        {
            _MyState = new MyTypeState();
            _MyType = new MyType(_MyState);
        }
    }

    /// 

    /// This is the original Bulky Class
    /// Some of its members moved UP (Wrapper)
    /// 
    internal class MyType
    {
        MyTypeState _MyState; //all fields now wrapped in MyTypeState
        MyTypeUtility _MyUtility;

        public MyType(MyTypeState myState)
        {
            _MyState = myState;
            _MyUtility = new MyTypeUtility();

        }
    }

    /// 

    /// All the fields moved from MyType to here
    /// 
    internal class MyTypeState
    {
        public MyTypeState()
        {
        }
    }

    /// 

    /// Methods moved here are stateless
    /// and/or if you add MyState, it can also be stateful.
    /// 
    internal class MyTypeUtility
    {
        public MyTypeUtility()
        {
        }
    }

Cleaning up

You now need to make sure that all is tidy. I usually do a mixture of the following.

1. Rename methods to suit the new classes
2. Split large methods at least into two
3. Rename classes, to make sure they make sense
4. Make RuntimeState, RuntimeExecutor classes private and check every method is private if its not called outside of its surroundings. If it is, I attempt to move the method to the utility class.
5. See if the Interface we made in Part 1, is still required (most of the time it wont be, only useful for refactoring)
6. If there are still large classes, I attempt it all from the start (Part 1)

.NET, c#, Methodology, Refactoring, Visual Studio refactoring, complex classes, code smells

Please Note: Please make sure you have proper unit test coverage on your classes, as you refactor.

Using Design Patterns which deal with compositional power, such as Wrapper, Facade, Adaptor, Bridge, Proxy and others are extremely useful for breaking large classes down. Rather have 10 smaller classes than 1 large one.

Well here is a very simple start to how to get your refactoring of that large class under way. Please be sure to compile your project after each step, to fix any errors, before moving to the next step.

[more]

Step 1

Split the class into two and create a wrapper.

1. Pick a very large, complex class. Mine is called "Runtime"
2. Refactor the name of the class. I will call mine "RuntimeExecutor". Add any suffix for the moment, you can always come back to it. (Naming classes becomes harder and harder as you divide classes.)
3. Add a new class with the "old name". Runtime.
4. Extract the Interface of ActionExecutor. In VS.NET you simply right-click on the RuntimeExecutor class, and select Refactor | Extract Interface
   For this exercise, simply select all the members.
5. A new class file with IRuntimeExecutor interface is created.
6. Now implement the interface on the Runtime class.
7. And create an Instance of the RuntimeExecutor from within the constructor of Runtime

It will look something like the code below

//abbreviated version of the class.
    public class Runtime : IActionExecutor
    {
        RuntimeExecutor _Executor; // this is the original class, renamed and now included as variable

        public Runtime()
        {
            _Executor = new RuntimeExecutor(); // we create the instance
        }

        #region IRuntimeExecutor Members

        public void Start()
        {
            _Executor.Start(); //we call the executor
        }

        public void Stop()
        {
            _Executor.Stop(); //we call the executor
        }

        #endregion
    }

Step 2

In the code above, you can see that I have added _Executor.Start() to the Start method. So in effect, Runtime is simple wrapping the Start of RuntimeExecutor. Now the next few steps are harder, but will increase the rewards.

1. Create a new class suffixed with State. I called mine RuntimeState
2. Take all the fields and properties for those fields from RuntimeExecutor, and cut/paste into State.
3. If you only have Fields, refactor them so that you now have Proeprties for those fields
4. Add a private field for the RuntimeState in RuntimeExecutor and
5. Modify the contructor of RuntimeExecutor by adding "RuntimeState state" as a new parameter, mapping the parameter to the field.

    //new state class
    internal class RuntimeState
    {
        private string _Field;
       
        public string Field
        {
            get { return _Field; }
            set { _Field = value; }
        }

    }

    internal class RuntimeExecutor : IRuntimeExecutor
    {
        
        private RuntimeState _State; //new field, replaces all the other fields that were here

        public RuntimeExecutor(RuntimeState state) //new param
        {
            _State = state; //set it and use it

            //now wherever it used the old fields, map it to the _State.NewField
           _State.Field = "some value";

        }
     }

So at this point, instead of 1 class, you have three. We refactored the class into two, the original and the State version. And we wrapped it higher up, by renaming it and substituting it for the new wrapper.

1. RuntimeExecutor (renamed original)
2. Runtime (new, a wrapper for RuntimeExecutor)
3. RuntimeState, (new split all properties and fields from RuntimeExecutor)

Step 3

We now need to split the original class further. This time instead of wrapping it, or refactoring UP, we now need to refactor down. Inspect the original class, and locate any methods that can be isolated and split from the class with minimal fuss.

1. Locate methods that are only looking at RuntimeState only, and move them, cut/paste into the RuntimeState class.
2. Make these methods internal, and point RuntimeExecutor to _State.MethodName
3. Locate methods that are utility functions to the class, such as getConfiguration, or IsRunning, and that can be moved to another class. If you have such functions, create a new class for it, and move it to the new location

So now you should have 4 classes from the original 1.

1. RuntimeExecutor (renamed original)
2. Runtime (new, a wrapper for RuntimeExecutor)
3. RuntimeState (now contains state related functions)
4. Utility class containing helpful functions

Conclusion

I follow this pattern for refactoring large classes that does not present itself clearly as refactorable. It gives me a set way of doing things, which then offers up more refactoring choices as I go along.
For example. When I find a lot of state, perhaps I refactor that too, into even smaller state classes.
Importantly though as you go along, you should be on the lookout for common classes. Perhaps another large classes State was extremely similar.
I then follow a pattern using inheritance for refactoring, and perhaps break things down even smaller.


In Part 2, I will show you how to utilize the wrapper for even more refactoring loveliness.

.NET, c#, Methodology, Refactoring refactoring, complex classes, code smells

Traditional Code Metrics such as LOC an CC is absolutely rubbish. It is pretty much as useful as Flesch-Kincaid is for reading. A C# code file measures quite readable.

[more]

Also, how strange that doing a search on Wikipedia for Code Metrics takes you to what can only be described as a Microsoft Advert. Its dying... It's dead... Only three or four add-ins for visual studio? Not much competition.

Does anyone think that Code metrics in the traditional form is helpful at all?

The problem is we can't measure code yet. We cannot make judgements on quality. or can we?

Perhaps we can look at code, from architecture. A Service Oriented Architecture has a different quality belief system than Object-Oriented. We know what formulae to create on either. We know that less state in a SOA application is more quality?

or what about patterns. If well known Design Patterns can be found within code, is it better than non well known Patterns, or no discernible pattern?

I think these sorts of questions can begin to get answered properly now. Someone will create that new system/formulae soon. We need a system, and it will be provided, eventually.

FeelGoodFactor

I follow the FeelGoodFactor principle. Requirements at first glance can be cloudy and messy. It can ambiguous and abstract, or it could just be plain daunting. However, you need to list what you can, and the attempt 'solves'. By solves, I mean a logical route to a solve, a prototypical solution to that problem, or at least common consensus by a few good men (cough). You need to get most of the areas in your head sorted, and then commit a bit of that solution in your head to paper and a review by your peers.

[more]
Here are some example of how I have conducted the last few projects, in the context of FeelGoodFactor.

Case Studies

Document Generator

Client wanted PDF and or possibly html versions of contracts and documents, downloadable from their website. They wanted the documents to be dynamic, so that it included data, their development team would inject into it.

There should be an editor for the mail merge like features, manage data templates (data fields) via XML, manage documents and map documents to data templates. A programmatic object model for adding data to the documents and rendering it out to pdf or html.

The FeelGoodFactor came once, I had established which PDF library, I was to use, and the approach I was to take. The approach was to create a parser, mapping expressions to data, and rendering to pdf, html or whatever the DocumentWriter was required. The extensible parts where DataFields (They would constantly change, or be added to), and the format the document would be required in, PDFDocumentWriter, HTMLDocumentWriter.

I was not concerned with the website, the document management side, as I knew I would quickly create a set of tables and UI of it, there was nothing to design for the early upfront stage of design.

I first wrote the XML Schema for the DataFields, and an ObjectModel to read/write the XML, and then moved to create a parser for html, to find my custom schema expressions.

So after 2 days, of writing code, aiding my design thoughts, and prototyping 3rd party technology, I quickly decided on a few Design Patterns that would coalesce into the base design, which I sketched in UML on paper and then started coding. Within 12 days, I finished the system, complete with +-70% test coverage, and example code in the form of recipes and Reference Help file for the small Framework. I then spent two days creating a few tables, and an API for Document Management on the server, using the Framework, ready for our website team, to use and integrate within the various sites and feature points.
They created a web service to wrap some of the functions and call it offsite, and it was done.

So the project had 2 days of thought, design, and prototyping, and 10 days developing. I developed in iterations of 2 days, and therefore had 5 iterations. iteration 1 and 2 was development and Unit Tests, 3 and 4 was refactoring and developing. Iteration 5 was testing and documenting.

Publishing Framework

For internal movement of files, website images, document management, it was required to move files of certain types configurable to different servers, folders, ftp and other locations. It also had to work work and accept ONIX request for books and publishable content, with managing files, such as TOCS, covers etc

After deciding on the architecture, Windows Services and Windows Communication Foundation, I created a small design prototype. I set to work creating the Web Service and Windows Service. Once that was communicating, sending any size files to the server, and not corrupting it or losing data, or anything else, I was in the FeelGoodFactor. I could then happily design each section within iterations as the project went on. The project also contained Onix publishing schema, and Onix Product publishing, with adaptors for DotnetNuke and many other facets.

This was developed over 7 iterations of 12 days each. I lead the design for 3 developers including myself. I spent the first 3 days, by myself, outlining some ideas, investigating Onix and other aspects. We had 2 meetings discussion ideas for the project. I spent 2 days creating an initial project, with hooks and areas for the team to continue on with, and gave them each a separate area to develop, with isolated objectives and isolated functionality. We integrated our work in iteration 2, and continued to code together in team system for iteration 2. We had regular scrums, not always in the morning and probably more every 2 days, than everyday.
I then did iteration 3 myself, as they worked on other projects. This gave me an opportunity to delete projects in team system (they get so messy), recreate projects, split and refactor projects, drag and drop classes aroundm change Namespaces, and create a more fine grained beast. With a more fine grained set of projects and solutions, it looks a lot better, and I got that FeelGoodFactor. At this stage with had 3 separate Visual Studio Solutions with around 6 projects in each.

In Iteration 4, I was rejoined by the two developers and we set to work creating unit tests, refactoring, and adding extra rounding features. Iteration 5, we started with a day or so of design meetings and requirements gathering, and set to work expanding the solution. I continued to design a new area, while the team continued developing new functionality.
Iteration 6 and 7, I was alone again, developing the last functions, refactoring, testing and making the design stick. I also documented various bits and pieces of the 'framework' portion of the project.

Iteration 8, which is the very next iteration, has been postponed, until our client, can meet and review. We shall not continue further, as specialization, further business rules and so on are required.

So the project has totalled 84 days, of which 10 days has been spent on design, thought and prototyping. However, I probably spend about 1 hour per day, refactoring and moulding the design, while adding new features.

The database, may get built slowly as we develop, but the focus is never on the database. I treat it as a throw away resource for these smaller projects. Both projects use reusable company Data Utilities, and can relatively easily be implemented in another database platform (depending on features, but we avoid platform specifics)

Design is not dead

Design is needed, you need to sit down and evaluate things. But the key is quiet time. Design must solve the problems it set out to solve. It must be a reachable design, within the time frame. You can reach a deadline easier, if your design is within iterations and gradually getting bigger, with the project. When you design upfront, one tends to add more things that are required. Designing for problems, evaluating solutions to problems, should be done upfront. Identify the problems within the system, and solve those, by careful thought, prototyping and any tool, such as UML, if it suits your needs, but don't design every object within your object model.

It would be a wasted of time, outlining the complete UML drawn object model, instead draw packages and the very important classes. Then create a set of rules/standards for the developers to follow, when creating the rest of the model. Also set a rule, that if you have to 'think twice', communicate with the team lead, to verify your thoughts. The same goes for when you cannot concentrate, perhaps never had breakfast. Get your team leader to pair program with you for a little bit. I call it The 'PushStart'. All of us need it from time to time.

So design until you feel that FeelGoodFator, and then develop an iteration, until it gets 'cloudy', then design until you get that FeelGoodFactor again. Now that is being Agile!

.NET, c#, Design, Design Patterns, Methodology, Visual Studio onix, xml, standards, uml, design, methods

No matter what software development methodology you adopt, you will always be faced with 'design' that is between good and bad, between necessary and unnecessary, between complicated and over-complicated. Design, even if it is technically good, can have disastrous effects on the project, if implemented incorrectly, or creates discord within a team, misunderstanding. In fact there are so many reasons, it would be better just to tell you what good design is. That's if I could, but I can't!.
However, I can say that Design is one of the most important factors for a project, it influences time lines more than anything else. Underdesigning, or overdesigning, neglecting to think ahead far enough, not including certain implicit requirements can all add days and weeks to a project.

Most of the time, its in your hands

A project requires thought and understanding. It takes a wide range of thinking outside of the box, thinking realistically, but yet being extremely open to ways of improvement. Here are three activities that will definitely aid your methodologies, skydiving and chess and music.

[more]

Skydiving

Skydiving, you jump out the plane, the ground is the deadline, and you simply ride the wind, and try to get finished before you hit the ground.

Often we are creating the projects, setting up a design, and coding it, within minutes or hours.
The first 'design' that sounds plausible, remains relatively unchallenged. We build some classes, think a little and then start coding. Most of the actual design is fleshed out as we go.
Unless you have a team, where design does get challenged, a lot. You get teams of developers debating direction and throwing pros and cons, which would be good, if there was a result. However, I have seen too many meetings end in stalemate, because it gets tiring. Not everyone thinks on the same wave length at the same time.

You cannot solve all the problems in one meeting. You cannot solve a design, or build some UML for the whole system on the whiteboard, using everyone's ideas, and thoughts. You will be there for days, weeks and many will get irritated and tired. It sounds obvious to let someone lead the design, but many projects, in todays flatten work environments want to get teamwork functioning and get the team to debate and meeting the designs. This will never work, unless it is very small, focused and within a time-frame, knowing that it may not be solved there and then.

software methodology principle number 1. Stop Skydiving.

Chess

Although we have refactoring tools, it takes real skill refactoring complex code, without breaking it, and still making it better. A large system can take weeks to refactor. In fact, most of the time, it doesn't get fixed properly. Maybe a little, but not completely.

So between "designing everything upfront" and "afterthough and refactoring", we need to consider more, and learn to read ahead. Like chess we should actually be stretching ourselves, practising to think ahead. In chess, its relatively easy to think of a few possibilities, but to really take advantage of the board, you have to be thinking ahead, further than your opponent.
In development, we need to get our read/think ahead skills better honed.

In chess, players think about the pieces, where each piece can move to, where it conflicts with moves of other possibilities. They construct a sort of tree, an algorithm, that each player might tailor to their own abilities and skills. They build this 'possibility tree', and then asses the tree for 'patterns', set moves, known strategies, and then further they inflate and deflate nodes in their minds eye, as to each nodes are more important, are more risky, or more possible to give them maximum rewards.

software methodology principle number 2. Play Chess.

Music

Rhythm, timing, tuning and arrangement, are the pinnacles of a good band. Without it, the best songs are rendered as noise.

The first thing is to find the rhythm section. Most importantly this will be your drummer and bass player. They have to be tight. They are providing the timing and the overall rhythmic feel. Then you need the rhythm guitar, who follows the drummer and bass, he has to add a little more flare at times, but overall, he is simply giving the treble feel. The Vocalists, and backup singers are providing the stage presence, the user interface and the meaning.
Developers who understand their position and work within their area, listening to the other band members and adjusting themselves to eachother.

Playing as a band, takes a little planning. Sometimes jamming is good, but most of the time, jamming will result in a rough product, perhaps with some awesome sections, and amazing solos, but as a record, it should be well rehearsed, tight and arranged with care. Then you get quality.

By the way, you also need someone to help you get into time, practice and coordinate things. There is always a band leader. There are also many many egos in a band, and the good ones learn how to do with that, so that the focus becomes the music, not the people.

software methodology principle number 3. Play as a Band.

.NET, Design, Design Patterns, Frameworks, Methodology, MethodologyWeek design, methodology, simple software design method

Oh what joy. Fons Sonnemans and Loek van den Ouweland have created a repository for ExtensionMethods. Go on, you know you want to. The site ExtensionMethod, allows you to submit your own creations to the community.

Now I had this idea, just around 10 minutes before I found the site, so they beat me fair and square. (By a mile). So therefore here is my top three ideas for the site, which I had, and hope they think about.

1. Grade it
Allow us to vote for it. However more than that, flag it with Compiled, Tested. Little icons that the site show next to each method.

2. Open Source the ExtensionMethod
Allow the community to edit the code, which will improve it over time. If I use it, and it fails because of something small, allow me to edit it, and solve the problem for everyone. Add original author, and contributor names to the list. Perhaps allow an 'Editor' per Type, to look after it and make sure its tested and complete.

2. Give us an Assembly
Run a build, once a month, compile all the snippets, which could be done automatically, if you get the correct data from the submitted snippet.
As part of the grading, is whether your automated compilation worked, when including the snippet. (snippet compiler like, compile on the fly and validate the method).

Now we can all download an Assembly for all these community driven ExtensionMethods. Updated: I found this nice, yet strangely odd set of ExtensionMethods for boolean.

.NET, c#, Design Patterns, Visual Studio extensionmethods, c#, vb.net

Software lifespan types

1. StaggeredRelease
   We develop versions and patches and roll them out periodically.
2. ContinuousLeak
   We fix and create mini patches, live alteration, component replacement, live into the environment. Typically its developed by in-house staff, developers and/or support.
3. OneHitWonder
   An App created and used, more or less with no problems or errors. Seems to be around for a while, and then fades away. Typically a utility, which has very little value, and soon gets replaced or irrelevant. Typically a quick utility written by someone on inspiration, but with little time to spend on its up keep.
4. RewriteEvolution
   An App released again and again, but always with the knowledge that it will be rewritten and/or always seems to get rewritten. Typically where there seems to be little justification on spending a lot of time on it.
5. ConcreteBeast
   An App, that is not liked or really wanted at all, but seems to be impossible to get rid off as too many things are dependent on it. Typically a piece of software installed before you got there.

[more]

Future proofing can be expensive, but there are some easy things to do, to increase your odds by a few percentage points. So in order to future-proof it, what should you do? First of all realize what kind of application its going to be, what characteristics will it have. You can use the quick list above, or some better scale.

Depending on less, not more

Loose coupling is an important factor in our making of agile software. Separating interface from implementation and other Patterns that promote loose coupling are very important for everyday development.

Use generalizations if you are not sure

Use a generalized, abstracted wrapper. Talk to your own Document object, and snap in with a Pattern of some sort, the actual real MS Word Document. This extensible model is vital, for allowing your Application to continue on through changes that happen around it. Actually this should be called use good Design Principles.

Control of Source and Components

Even though one tries to depend on less, we may still use many components, user interface controls and so on, which is good for reuse, but bad for dependencies. But try to avoid using reusable components, especially those you do not own, which is in direct conflict with increasing reuse and everything I stand for, but nevertheless.

Choose the right platform

Sometimes the platform or framework may dissipate. Choose your platform wisely. Think of Linq2SQL. Sometimes you need to go with something, like DAO, because, well, what else is there, that you could trust more? Now its ADO.NET which us currently 7? years old.

Know your RoadMaps

Get all the roadmaps of technologies that you depend on. Understand the manufacturer, their product cycles, and history. Learn a lot, read a lot and get your intuition firing.

Ask your manufacturer

How long do you expect to support this component? How long? Its ok if its only 3 years, if it coincides with your application lifespan. Sometimes they can be very helpful, and sometimes you will find subtle aspects that will help your choice, so just ask.

Avoid the Latest thing

Unless, your intuition says otherwise. Its sometimes great being one of the bleeding edge, but its very risky.

Design, Methodology

We all know that the learning curve for Frameworks are quite high. We all know that Frameworks demand commitment, and learning time. We know this because we struggle with it everyday, whether its Java or .NET, Swing, or MFC.

1991-1993, Ralph Johnson and others wrote about Recipes and Cookbooks. There have been papers on documenting frameworks for more than 15 years. MSDN has some of the flavours but none of it is actually polished, nor tailored to the audience and the way in which people seek information.


My D4 Software Frameworks Pattern Language points you to some Patterns for documenting Frameworks:

WritingReference
You must provide a reference of interfaces.
WritingRecipe
You need to provide examples of isolated features of the framework that a pure reference cannot show.
WritingCookbook
You need to combine a large amount of recipes.
SelfDescribing
Using tools and generators, you can generate documentation.


All of them are self explanatory, except let me add, amazing that blogs seem to be the number 1 source for this kind of information, as well as google tends to find things on MSDN better than MSDN finds on their own material?.

The Solution to the MSDN Dilemna

The last one (SelfDescribing) is where I am particularly keen. Not GeneratingReference but SelfDescribing Imagine if a method on a class called MethodA() had been marked with an attribute called TemplateMethod()

Well a Tool, would be able to highlight it for you. If you had a tree of all Types, TemplateMethods and Hooks, you would now better understand the way in which the types where constructed. You would now know that MethodA() called Method(B) then MethodC(). The internals of MethodA() ois not known, shown or cared for. But from a documentation point of view it is crucual for certain designs.

It is this, that is missing from MSDN, because it is missing from all frameworks. Documentation needs to be better, but cannot, until the compiled code, can be examined in a much more thorough way. Developers of Frameworks, must declare their intentions of methods, class and patterns by attributing metadata onto it.

So the solution, is to mark the .NET Framework Types with documenting metadata. Allow all Framework developers to utilize this metadata descriptions for their frameworks, and provide a tool to inspect it.

.NET, Design, Design Patterns, Frameworks, Visual Studio msdn, documentation, frameworks

Effectively, here is an Index to the pattern language in draft.

A Pattern Language for Designing, Developing, Documenting and Deploying Object-Oriented Software Frameworks

[more]

Design, Design Patterns, Frameworks, Methodology frameworks, design patterns, coalescent patterns