JP Labs

News time

2010-12-29T18:35:00.000-02:00

News time, children!

My blogging activity might be low recently, but there are two news worth mentioning.

Fisrt, the repository move. I've been trying Mercurial for some time. It's time to admit I like it. I you are interested in following the code I write, from now on you'll have to look at my Mercurial repository at Bitbucket.

I know that Google Code has Mercurial support, but I simply preferred to go to another site. One advantage of using a distributed version control system is that I can move hosts any time I want, without loosing anything. Anyways, my experience with Bitbucket has been very good so far.

Second and last, my blog url has changed. You might already have noticed, but you're now at http://jplabs.bochi.it/.

Thanks for listening! And now, some music.

A Tale of Singletons and Global State

2010-10-15T10:24:00.001-03:00

Some weeks ago, I received one odd feedback. I was talking to this guy and he said "A fellow of mine read in your blog that you are in favor of singletons. Is that true?" I remembered that the post was ultimately about Design Patterns. I used some extrapolations of the Singleton Pattern to get to Dependency Injection. I never supported singletons. I just used it as a very simple and known pattern to get to something bigger. I was so astonished by the question that I couldn't answer it. At the time, I was not able to support my real opinion. Today, I'll try to express myself clearer.

According to wikipedia, the singleton pattern is a design pattern used to restrict the instantiation of a class to one object. This is useful when exactly one object is needed to coordinate actions across the system. In the usual way it's implemented, there's a private constructor and an static property or method that always returns the same instance. There are a few big problem with such a class. One, it violates the single responsibility principle. A class should not be responsible for its own lifetime. Two, the singleton instance is attached to the global state.

It happens that both these problems can be eliminated and we still have a singleton. The creation of the singleton could be delegated to a factory. And the factory doesn't have to be attached to the global state. That's what differentiates a good singleton (one instance per factory) from a bad singleton (one instance per AppDomain or JVM).

I have seen a good deal of bad singletons in projects I've worked on. They caused me pain. Many times, I was forced to convert some of them into something nicer. I know how hard they are to test, and how they tend to create hidden dependencies. I don't like the bad singleton. Some people even say they are evil. Nevertheless, they are ubiquitous. Just for the sake of the example, let's consider the .NET framework. Here is a list of singletons (and static functions that access some form of global state) built in the framework:
System.AppDomain.CurrentDomain;
System.DateTime.Now;
System.Environment.*;
System.Globalization.CultureInfo.CurrentCulture;
System.IO.File.Open(...);
System.Random.Random(); //uses System.Environment.TickCount
System.Threading.SynchronizationContext.Current;
System.Threading.Thread.CurrentThread;
System.Threading.Tasks.Task.Factory;
System.Transactions.Transaction.Current;
System.Web.HttpContext.Current;
System.Windows.Application.Current;
System.Windows.DependencyProperty.Register(...);
System.Windows.EventManager.*;
System.Windows.Forms.Application.*;
System.Windows.Forms.MessageBox.Show(...);

Of course, .NET is not an exception. All the main platforms we use today have quite a few examples like those. Do you know any singleton-free programming platform? The only one I've heard of is Newspeak. Why aren't there more languages like that? Why do we use programming platforms so packed with vicious singletons and APIs that access global state? I don't have a complete answer to these questions, but one thing is certain. There must be a good reason for them to be there. The cost of removing all singletons probably doesn't pay off. In fact, I think it's close to being impossible.

Let's consider the computer's clock. In .NET, we can read the current time by reading the DateTime.Now property. Although it's not a classic singleton, it's completely equivalent to one. It could be like that: Clock.Instance.GetCurrentDateTime(). The real problem here is that we are accessing global state. Every time we read the value, it changes. Now, is that such a bad thing? Of course not. A clock is supposed to move without being told so. A stopped clock is useless. Would you blame the framework designers for coding a singleton in such a special case? I wouldn't. There's usually only one physical clock available in a machine anyway.

What about a logical clock? I needed one once. The application I came to work with had a rule engine. Many of the rules were dependent on the current date and time. For example, the customer would be eligible for a fee waiver only if his/her last payment were due for less than a couple of days. No surprise, the team found out that rules like that were the hardest to test. The only way to fake the system time is by changing the computer's clock. However, it affected the application in other unforeseen ways, and that was not a viable solution. Not being able to test all the rules, the original team did the best they could with the time that was given to them. They deployed the application without testing everything.

Nonetheless, the situation can always get worse. Another problem appeared. We realized that many users were working in a different time zone and that the application was using the machine's local time. It was supposed to use US Central Standard Time. Compensating the local time zone wouldn't fix it because some users had an incorrect time zone set. And the users could always fiddle with the local clock. In the end, we simply couldn't trust the system's time.

The solution? I created a logical clock that gets synced with a trusted server every now and then, completely independent of the local time. I just provide the rule engine with a proper clock and the time zone bug was fixed. Since there's only one synced clock per rule engine (and one rule engine per application), the clock is a singleton. A good one though. Good enough to have one additional benefit. We could now test the time-dependent rules easily. All we need to do is to mock the clock.

So, what's my take on singletons? They are a necessary evil that has to be understood and handled with care. If you really need to create one, make sure it's a good one.

Tweaked Events at CodeProject

2010-09-09T18:50:00.000-03:00

Two days ago, I published my first article at CodeProject: Tweaked Events (ex-"Custom Events"). Go check it out! :)

On Design Patterns and Dependency Injection

2010-05-19T12:39:00.003-03:00

Since I heard of design patterns for the first time, there was one aspect of it that puzzled me. They seemed to be universally considered a good thing by itself. There seems to be a common belief that the lack of patterns in a program is a bad sign. Or, conversely, the ubiquitous presence of patterns in a software code is a good sign.

I never agreed to anything like that. Ideas like that go against my instincts. More specifically, the instinct to avoid recurrent code patterns (in the dictionary meaning). They are bad for maintenance, to mention one thing. Well, the so called design patterns are exactly that, recurrent code patterns. So, why are they are so popular? The reason is that their basic idea is mostly misinterpreted.

A while back, I read a very enlightening text written by Mark Dominus entitled Design patterns of 1972. He starts by saying:

"Patterns" that are used recurringly in one language may be invisible or trivial in a different language.

And concludes affirming:

Patterns are signs of weakness in programming languages.

I agree with him. And so did most of the people that read and commented about the article. Ralph Johnson (one of the authors of the ”Design Patterns” book) wrote a reply. And Mark wrote a reply to Ralph’s reply. I think there are some ego sparks here and there, but they generally agree with each other. If you are surprised by their agreement, you definitely should read their texts.

Creational Patterns and Dependency Injection

Despite their value, I will not simply quote other people’s opinions. I have a point to make, too. I’ve been reviewing the whole subject recently, and an idea came to my mind. It’s not very innovative, but it’s worth phrasing anyway.

All creational patterns (namely Abstract Factory, Builder, Factory Method, Prototype, and Singleton) are no more than specific applications of dependency injection.

I’ll try to build my case around the singleton pattern. It’s possibly the simplest pattern, and the easiest one to understand. To begin with, I do not want to advocate it as bad pattern. For many simple situations, it fits the problem perfectly. I see it as an extreme simplified solution to a possibly more complex problem. It has drawbacks, though. Some cases require something similar, but just a little bit more sophisticated. I’ll go over some possible modifications to the singleton pattern through the rest of this article.

Before moving on, a comment. I’ll use some C# for my examples below for a single reason, it’s the language I know the most. I’m sure the general idea is valid for any language.

- Global State and Object Life Cycle

Everyone knows that global state is evil. The strongest argument against singletons is that they bring create global to an application. Depending on global state has consequences.

Suppose you have a singleton being used a multi-threaded application. A reasonable assumption is that the singleton object has to be thread-safe, i.e., it has to be able to be accessed by different threads without misbehaving. That’s not completely necessary.

One alternative is having a not-so-singleton object that has one instance per thread instead of one instance per process. This can be easily achieved with thread-local storage, which is supported by most programming languages. Even though these thread-local singletons are not valid to every problem, they are the easiest and safest way to have a thread-safe singleton.

We can go deeper on the idea of multi-instance-singletons (by the way, the term “singleton” might not be appropriate at this point, but that’s just a matter of naming). In many cases, what you really need is one instance per “context” or “call”, or whatever. A nice .NET example is on the System.Web.HttpContext class. It has a static property called Current that return an HttpContext. This might not be commonly seen as an use of the singleton pattern, but it’s a very close variation. The way I see it, it’s basically a singleton that has one instance per http request.

Controlling the life cycle of “singleton” instances in these complex situations can be too complicated, and too specific to still be considered a pattern. That’s not actually a problem. I’ll get back to the subject later.

- Abstraction and Decoupling

Now, I want to expose a different facet of the pattern under discussion. Usually (at least in my code), the exposed type of the singleton instance is more abstract than its actual concrete type. Something like the C# code below.

public static readonly IAbstractThing Instance = new ConcreteThing();

You may reasonably argue that this is not a good usage of OO. If IAbstractThing were destined to have only one concrete implementation, then you wouldn’t even need the interface to start with. But, in real world applications, that’s commonly not the case.

Exposing an abstract type opens some possibilities. You could, initially, delegate the responsibility of creating the instance to another component. By doing that, you achieve more than one advantage. You are now close to being able to create mock objects for unit testing. A secondary (or should I say primary?) advantage is better decoupling. The component that exposes the instance does not depend on any concrete implementations anymore, i.e., there will be one less dependency around

A group of components that work in this way (one component exposes an abstract object, and another creates the object) has dependency injection in it, even if no DI framework was used.

- Dependency Injection

Of course, the more sophisticated you get, the farther you get from the original pattern. All the intermediary variations are valid solutions for real problems. At each point, the decision of whether improve the code or not is a matter of balancing the costs of development (including test, support, etc.) with the actual advantages it will bring to the application.

The usual shortcut through such evolutionary process is a framework. And there is a lot of stable and dependable DI frameworks free for anyone to use. They are capable of providing a much more powerful solutions than any of the creational design patterns can, and at a lower cost. As I mentioned earlier, life cycle management might get very complex. Not with a good DI framework. Most of them support a broad range of extendable life cycle styles. By the way, my current favorite is Ninject. If you are a .NET programmer, I recommend that you give it a try.

Final Remarks

If you are building a non-trivial application and find yourself using a big mix of abstract factories, builders, prototypes, or whatever, be sure of one thing. You are in need for a good DI framework. Any language that has built-in DI, or has a built-in DI library, or enables such a library to be build, do not need to depend on “creational patterns”.

A similar statement can be made to other types of design patterns. Consider the strategy pattern, for example. It’s trivially implemented using first-class functions (C#’s anonymous methods). The observer pattern can be effortlessly implemented with C# events. The iterator pattern can be graciously implemented with the IEnumerable interface and some help from the foreach keyword. Using the yield return keyword, it’s possible to elegantly represent complex object interactions with a only a few lines of code.

Please, don’t get me wrong. Design patterns are good in many ways. For one thing, they provide a common vocabulary to programmers going through similar problems. What I’m saying is that they are generally misunderstood and overrated. Their ultimate purpose is to become invisible. Do not think design patterns are timeless. They aren’t.

Lock-free Synchronization in C# 4 Events

2010-03-15T18:57:00.008-03:00

If you're keeping yourself up-to-date with .NET Framework 4.0 and C# 4, you might have read "Events get a little overhaul in C# 4" series on Chris Burrows' blog. He's explaining some changes to the intrinsic implementation of events. They were introduced in C# 4, which is in RC now. One important detail is that there are breaking changes included. So, you'd better get familiarized with that. If you haven't read his latest posts, please, do it now.

(intermission)

Welcome back. If you are fond of details like me, you might've gotten curious about that "lock-free synchronization code" used to add (and remove) event handlers in C# 4. That's exactly what I intend to show here.

The first step is simple. With a little help from Reflector, I disassembled the code generated for an simple event. After some variable renaming, here's what I've got:

private EventHandler dlgMyEvent;

public event EventHandler MyEvent
{
    add
    {
        EventHandler valueBeforeCombine;
        EventHandler valueBeforeAttribution = this.dlgMyEvent;
        do
        {
            valueBeforeCombine = valueBeforeAttribution;
            EventHandler newValue = (EventHandler) Delegate.Combine(valueBeforeCombine, value);
            valueBeforeAttribution = Interlocked.CompareExchange<EventHandler>(ref this.dlgMyEvent, newValue, valueBeforeCombine);
        }
        while (valueBeforeAttribution != valueBeforeCombine);
    }
    remove
    {
        // code ommitted (it's too similar to 'add')
    }
}

This code isn't obvious to understand (at least for me). I took quite some time to figure out what was happening. Of course, variables like handler2 and handler3 don't help much. I spared you from the trouble and renamed them.

Looking carefully, the secret ingredient is the Interlocked.CompareExchange. Since most people are not familiar with this function (I wasn't), let me explain it. An equivalent implementation would be like the code below. There's a very important difference, though. The real one runs as an atomic operation.

public static T CompareExchange<T>(ref T location, T value, T comparand)
{
    var previousValue = location;
    if (location == comparand) location = value;
    return previousValue;
}

Now, we have everything needed to comprehend the idea behind of the new event implementation. To make it crystal clear, let me expose it in human language. It goes like that:

Copy this.dlgMyEvent into valueBeforeCombine;
Create a new delegate called newValue by combining valueBeforeCombine with the supplied value;
Atomically, verify that dlgMyEvent is equal to valueBeforeCombine, and overwrite dlgMyEvent with newValue;
If the overwrite didn't happen (i.e., dlgMyEvent changed sometime between step 1 and 3), go back to step 1 and try everything again;

When I finally visualized this pattern, it felt like an epiphany. As a computer scientist, I'm ashamed of not knowing it before. Now, I know it's called compare-and-swap. Obviously, it can be used a other scenarios, but all this is very low-level coding. So, kids, don't try this at home. :)

I've a final comment. I decided to imitate MS and implemented a similar synchronization mechanism in my Custom Events. Here's the latest source download link.



My Extension Method Colllection
2010-01-12T11:46:00.004-02:00
In addition to the small components I've shared in this blog, there's a lot of short functions that I've collected over time. Most of them are not interesting enough for a post of their own. Actually, I never seem to find a proper place to store this stuff. Until recently.

I realized that most of these functions were already coded as extension methods. And the ones that weren't extension methods, could be refactored to be. So, I started to put all of them in a single assembly called JpLabs.Extensions.

Some of this methods were extracted from sources like this question at SO, but the majority were written by me (or a co-worker) for direct use in other projects.

The full source is available to download here. Feel free to use it as you see fit. All feedback is welcome.


Iterators and Argument Validation
2010-01-07T11:02:00.011-02:00
Iterators were introduced in C# 2.0, so they are not new. Before LINQ was born, I think most C# programmers had never seen an yield operator. I'm not sure when I saw it for the first time, but I certainly wasn't an early user. I still remember the puzzlement I felt trying to understand how this stuff works.

If you ever opened the Enumerable class in Reflector you might have noticed that it has several nested private compiler-generated classes. This nested classes are generated whenever a class have iterator functions.

The point I want to make in this post is about argument validation. I've seen in more than once people making a simplification that, IMHO, should be considered a defect in production code. As an example, consider the code below (which I extracted from an answer in stackoverflow).
public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(
    this IEnumerable<TFirst> first, IEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> selector
) {
    if (first == null) throw new ArgumentNullException("first");
    if (second == null) throw new ArgumentNullException("second");
    if (selector == null) throw new ArgumentNullException("selector");
    
    using (var enum1 = first.GetEnumerator())
    using (var enum2 = second.GetEnumerator())
    while (enum1.MoveNext() && enum2.MoveNext())
    yield return selector(enum1.Current, enum2.Current);
}

It's quite simple and it might be hard to find any issues even after reading it carefully. It will even pass through positive testing. The problem only appears when you try to send a null value to any of the three parameters. As you may know, the code of an iterator function is not executed right after the call. It's executed only when the IEnumerable is enumerated. This behavior is called delayed execution. Because of that, when a null argument is passed, the ArgumentNullException won't be thrown until the enumeration of the returned value. This is highly undesirable.

Fortunately, there's a very simple way to avoid the problem. As mentioned in MSDN community content in this page, you should break the iterator function in two functions. Like this:
public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(
    this IEnumerable<TFirst> first, IEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> selector
) {
    if (first == null) throw new ArgumentNullException("first");
    if (second == null) throw new ArgumentNullException("second");
    if (selector == null) throw new ArgumentNullException("selector");
    
    return Zip(first, second, selector);
}

private static IEnumerable<TResult> ZipIterator<TFirst, TSecond, TResult>(
    IEnumerable<TFirst> first, IEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> selector
) {
    using (var enum1 = first.GetEnumerator())
    using (var enum2 = second.GetEnumerator())
    while (enum1.MoveNext() && enum2.MoveNext())
    yield return selector(enum1.Current, enum2.Current);
}

I highlighted the key points. Notice that only the second function is an iterator, and it's private. The only place where it should be called is in the first function.

By the way, Zip is an extension method added in .NET 4.0. If you want to read more about Zip and the elusive behavior of exceptions in iterator functions, I recommend this two posts:

- Bart's LINQ's new Zip operator at http://community.bartdesmet.net/blogs/bart/archive/2008/11/03/c-4-0-feature-focus-part-3-intermezzo-linq-s-new-zip-operator.aspx;

- Eric's Zip Me Up at http://blogs.msdn.com/ericlippert/archive/2009/05/07/zip-me-up.aspx;

- Eric's High maintenance at http://blogs.msdn.com/ericlippert/archive/2008/09/08/high-maintenance.aspx.


Enhanced Weak Events Part  Two - Immutable Event Entry List
2009-07-30T10:03:00.018-03:00
Posts in this series

- Part One: DynamicMethod vs. Expression Tree Compilation
- Part Two: Immutable Event Entry List (this post)
- Part Three: Synchronized Events (to be written)
- Part Four: Custom Events and unsolved problems (to be written)
- Dynamic Code Generation With Lambda Expressions (to be written)

C# Events - Basics

Just for a start, let me reinforce some preliminary definitions in an very factual way.
Delegates are tuples composed by a reference (A) to a method, and a reference (B) to a target object.
- All delegates are made immutable;
- The reference (B) will be null if the method is static;
- When a delegate is invoked, the referenced method is called. If the method is an instance method, the target object reference is used;
Multicast Delegates encapsulate a collection of delegates.
- When invoked, all of its delegates are invoked in an unspecified order;
- Multicast delegates can only be created by composition or decomposition. Composition is the merging of two delegates into a new multicast delegate. Decomposition is a subtraction of one delegate from another;
- There's no such thing as an empty multicastdelegate. If a decomposition would create one, the result is null;
- For all purposes, a multicast delegate is a delegate;
C# Events are nothing more than a special type of property that have add and remove accessors instead of get and set.
- Only events of a delegate type can be declared;
- Due to the special accessors, only the class that owns the event can reset its value or invoke its delegate, if it's not null;
- When add is called, the current delegate is composed with the delegate that was provided;
- Conversely, remove does a decomposition;

C# Events - Not So Basics

Having said that, let's dive deeper. When it comes to c# events, there are two interesting problems that can be easily overseen. The simpler one is that an event is always initialized with null, and they can always be decomposed to null later on. In other words, an event can be null at any time. Since it's not possible to invoke a null delegate (without causing a NullReferenceException), any class that provides an event has be to sure that the event is not null before invoking it.

The other problem appears in multithreaded applications. A thread might be adding or removeing an event handler while the event is in the middle of an invocation. This is a race situation, and has to be correctly handled. If you want to check a complete explanation on this subject, I recommend you to read Eric's post on events and races.The generally recommended solution to both problems is the following pattern:
var handler = this.MyEvent;
if (handler != null) handler(this, eventArgs);

Now, let's move to main subject of this post.

Mutable Event Entry List

As you may already have noticed, my custom events were born from Daniel Grunwald's WeakEvents. Internally, to represent each event entry* he chose to use a List<EventEntry>. Using regular a List<T> seems very convenient. It will make add and remove operations simple and quick to execute, but there is a downside. List<T>'s are mutable.

One of the major disadvantages of mutable objects appear on multithreaded applications. Consider the problems that I've introduced above. They apply to custom events too. Alas, that invocation pattern can't be used.

One goo point is that even an empty WeakEvent will never be null, so we don't event-is-null problem. The concurrency problem, though, can't be avoided. As the event entry list is mutable, any access to it should be synchronized if we want to make the custom event thread-safe.

Looking at Grunwald's code that makes the invocation of a WeakEvent, I noticed that the entry list is copied to an array before invoking the entries. No locks are made. Is that enough? Unfortunately, the answer is no. Copying the list to an array is not an atomic operation. If the list is modified in the middle of the copy, things will probably get messy. Even if he had enclosed the copy in a lock block, there would still be the performance problem caused by the copy itself.

* An event entry is an abstraction of an event handler delegate. Weak event entries, for example, serve the same purpose of a regular delegate. The sole difference being that it does not keep a strong reference to the target object.

Immutable Event Entry List

Naturally, my suggestion is to use an immutable list to store event entries. By the way, as I mentioned in the beginning, all delegates are immutable. And regular c# events use immutable multicast delegates internally. There's probably a good reason they did it that way, and I don't see why custom events should differ. Let's consider some aspects of this decision.

Lower invocation overhead: There's no need to copy event entries before invoking them because the list does never change. All that's needed before the invoking is to read the current list object and store a reference to it in a local variable. If the event is changed by other thread, a new list will be created, but the invocation will be working on an untouched list object.

The only possible problem may happen when an handler is removed, but an invocation is being made. The result is that the handler is invoked even after it has been removed from the event. Actually, this is a third interesting problem. The post from Eric Lippert (events and races) that I've suggested earlier talks about this stuff too. If you read it, you'll see that my custom events  are actually imitating the exactly same behavior that regular c# events have.

Unfortunately, there is also a trade-off involved. Add and remove operations will be inevitably slower. Each time they happen, a new list has to be created, and a partial copy of the current entries will be made. Personally, invoking performance has higher priority than adding or removing performance, so I think this trade-off is acceptable. If anyone disagrees, I'll be very happy to hear your thoughts.

Thread-safety: Invoking an event when its entry list is immutable completely thread-safe without any need for locks. That's a great win.

Add and remove, on the other hand, are not thread safe. I can say as an excuse that the mutable-list-approach is not thread-safe too. At least, I did not created a problem that was not there before.

Looking at regular events, we can see that even they are not thread-safe when it comes to add and remove. Would the C# language team let so big a hole in the language? Of course not. The solution is hidden from the programmer's eyes. Whenever you declare a regular c# event, the compiler generates the two accessors (add and remove). Both of them are them decorated with this custom attribute:[System.Runtime.CompilerServices.MethodImpl(MethodImplOptions.Synchronized)]. You can make a test assembly and decompile it to see. Or you can read this old MSDN article too.

So, in my opinion, locking should be avoided unless there's no alternative. In a reusable component (like custom events), it's preferable to let it be not thread-safe and free of locks than forcing an overhead that is not always required. A developer that uses your component should be able to use any synchronization mechanism he wants, and only if needed. Inside the .NET Framework, almost all collection implementations are not synchronized for this reason. Following the same pattern, I won't put any locks on my Custom Events. User developers should be aware of that, and put locks around add/remove it they need it.

Conclusion

In a world where multi-core processors are everywhere, multithreading and immutability are increasingly important. If you want to read more about immutability in C#, you might want to check this out.

If you read this far, you probably are interested in events. A very extensive article you might want to read is on codeproject.

Finally, as usual, here is the download link for the latest version (v1.1) the my CustomEvents' source code. See ya!


Enhanced Weak Events Part One - DynamicMethod vs. Expression Tree Compilation
2009-07-14T16:20:00.015-03:00
Introduction / Background

In my last post some months ago I showed a use case for weak events*. Since then, I've ran into some other interesting event problems, and that gave me time to make some improvements over Daniel's code.
* If you're not familiar with the concept of weak events, I believe the best introduction you can find is Daniel Grunwald's article.

Weak Events Enhancements Part One: Faster Forwarder Delegates / DynamicMethod vs. Lambda Expression Compilation

The first improvement that I made is, in fact, the last that I've implemented. In Daniel's final solution (FastSmartWeakEvent), a forwarder method is compiled using System.Reflection.Emit.DynamicMethod. Although this is a very light way to generate code dynamically, it's very low level. Usually, the generating code is quite unreadable and very hard to debug, to modify or to extend. Daniel's first version, for example, had a type safety issue. I imagine that it was hard to find, and the solution is not obvious to understand (at least it wasn't obvious for me).

For some time I was wondering if there was any higher level alternative to a raw DynamicMethod. The usual way to hide low level code is by encapsulating it inside a higher level framework. I don't remember how the idea came to me at the time, but I realized that .NET Framework 3.5 already contains what I was looking for: Expression Trees.

Expressions trees can be composed dynamically, and compiled to a delegate whenever they are ready. I tried it out, and developed a new WeakEventForwarderProvider class. The implementation details have several interesting catches. In fact, I think there are enough of them that I'll cover them in another post. Until then, feel free to check out the code. For now, I'll share the results that I've found.

Maintainability: Readability is a key feature of a maintainable code, but opinions might always diverge about it. It depends on coding style, and my style is somewhat unusual. Even though I admit the new code is not easy to read, I feel that it has improved a bit from the original; and there is still room for improvement. Debugging, on the other hand, has certainly been improved. The main reason for this is that the debugger is able to show any intermediate expression as nice programmer-readable strings. They can also be converted to string. So, it's much easier to know what is being generated.

Correctness: Using DynamicMethod, it's possible to generate any sequence of IL commands. One obvious advantage of this is a very high flexibility. The trade-off is that it's too easy to generate broken code. Expression trees work as middle ground solution. Its major restriction is that statements are not allowed inside expressions. Fortunately, that can be worked around, specially when generating small pieces of code (I promise to explain how in a future post). A great advantage expressions do provide is a very good validation at dynamic-code-generation-time. The type safety bug that leaked from Daniel's first version of WeakEvents was found and fixed very easily using expressions.

Compiling Performance: I knew that expression compilation uses DynamicMethod internally, so I took for granted that it would be slower to compile. Surprisingly enough, the results that I've measured were close to a tie. Sometimes it appeared that lambda compilation was faster. This is still puzzling to me.  While I don't find a good explanation, I'll let the topic open for discussion. My best guess is that the overhead that expression trees add is so much smaller than the overhead of emitting the code that, proportionately, the first can be disregarded.

Executing Performance: Code execution performance is a tricky subject. You can always try to tweak assembly code to make it faster. You can do it at IL level with DynamicMethod too. But that generally is a bad idea. To quote Donald Knuth, premature optimization is the root of all evil. I've found out that the expression tree compiler is quite good. My tests have shown that my expression-generated delegates were about 15% faster to execute than the ones that Daniel has got. That's better than I've expected.

Performance-wise, I came to a conclusion that simply reinforces one of my software beliefs. A well-built framework will not make user code slower. Actually, some remarkable cases are capable to make user code even faster. In my opinion, instead of optimizing your code, find a good library or framework to do the job for you. If you can't find it, do not optimize. At least not now. Considering all aspects, my expression tree solution was very satisfactory. And it was fun to code too.

Following Posts

These are the next posts that I'm planning to write on Weak Events:
- Events Part Two: Immutable event entry list;
- Events Part Three: Synchronized Events
- Events Part Four: Custom Events and unsolved problems
- Dynamic Code Generation With Lambda Expressions

Source Code

Download Custom Events source code here (v1.1 @ 2008.07.31). It contains stuff that I've not discussed yet. If you're eager to see it, feel free to delve into the code. See you soon (I hope :)).

Updated at 2009.08.03: Source code link updated.


Trackable Properties with Weak Events
2009-02-18T18:30:00.036-03:00
Coming back from a long period of silence, I'm posting a little piece of code that I used in a project of mine. Usually, I start by the problem that I was facing and them I move to the solution. This time, I'll try another path.

TrackableValue<T> - a very short description 

TrackableValue is a generic class that encapsulates a "value" and provides an event that will fire every time this "value" is changed.

Inspiration

When I came with the basic idea of TrackableValue, I was learning WPF. More precisely, I was trying to get a grip on Dependency Properties. This is a very complex concept to get used to straightaway. To be short, they are at the base of several of the main advantages of WPF. Stuff like data biding, animations, themes, styles, property inheritance through WPF trees, and others are only possible because of dependency properies.

At a time, I saw the advantage of creating a lighter version of these super properties. There were 3 requirements that I was seeking to fulfill. One, it must not be dependant on the WPF assemblies or any other UI framework. Two, it must be trackable for changes. Three, it must use some kind of weak events (more on this later). Obviously, the code that I'm posting fulfills these requirements.

Motivation

One advantage of a value being trackable is that you can make a one-direction data binding. In a Model-view-controller-based application, for example, binding between a model and a view is almost imperative. UI and binding are too often seen working together. Most UI frameworks already have support to binding and I don't intend to substitute that. I didn't want my component to be dependent on any UI. Also, it shouldn't be directly associated to binding. Data binding is just one possible usage of it.

My purpose was simply to facilitate the task of creating solid value-changed events. Putting code on the setter method of a regular CLR property is a simple and very straightforward way to track changes on a property. So, what is the problem with this approach? The problem is that it does not favor composition. I'll explain.

When you want to track changes from outside the class that owns such property, things start to get complicated. Generally, it's possible to make a subclass that overrides the property, but it's not a good policy to let it be the only way of extending your application. Essentially, I'm repeating the mantra "favor object composition over class inheritance". I think TrackableValue is a fine attempt accomplish this. Also, it has the advantage of using weak events, which I'll explain later.

Class "interface"

I was never very fond of UML, so I prefer to show a class structure in pseudo-code than using a class diagram. Here's an interface-like description of TrackableValue:
public sealed class TrackableValue<T>
{
    /// <summary>Constructor</summary>
    /// <param name="owner">The object that 'owns' the value, if any</param>
    public TrackableValue(object owner);
 
    /// <summary>This is the actual value</summary>
    public T Value { get; set; }
 
    /// <summary>This is the event that will be raised every time a new value is set</summary>
    public event EventHandler<ValueChangedEventArgs<T>> ValueChanged { add; remove; }
 
    /// <summary>This property is only used to set the 'sender' parameter to the event handler</summary>
    public object Owner { get; }
}

Now, an example. The code below is a simple class with a trackable property using a TrackableValue.

public class ClassWithTrackablePropery
{
    private readonly TrackableValue<double?> innerValue;
 
    public ClassWithTrackablePropery()
    {
        innerValue = new TrackableValue<double?>(this);
    }
 
    public double? Value
    {
        get { return innerValue.Value; }
        set { innerValue.Value = value; }
    }
 
    public event EventHandler<ValueChangedEventArgs<double?>> ValueChanged
    {
        add        { innerValue.ValueChanged += value; }
        remove    { innerValue.ValueChanged -= value; }
    }
}

There isn't much to explain about it, but I would like to make a few remarks. First, observe that I've set the innerValue member field to readonly. That's because the actual instance of TrackableValue should not change for the lifetime of the instance of ClassWithTrackablePropery. Second, when instantiating the TrackableValue, I'm setting its owner to this. This way, the sender parameter sent to event handlers will be the ClassWithTrackablePropery object that owns the value that changed. This might be useful in a code that uses this class.

Weak Events

Last but not least, I would like to introduce the notion of weak events. Right now, you might think that my code is too simple and didn't even deserve to be posted here. I wouldn't blame you to have this opinion. I tell you that the really sexy part of TrackableValue is the use of weak events. Before showing what they are (if you don't know already), I'll bring out a situation.

Suppose you have three components in an application: MyView, MyBackEndService, and MyController. MyController controls the application. It manages intances of MyBackEndService and MyView. MyBackEndService communicates with some external application and may change its state asynchronously. MyView is an windows forms window that shows the state of a MyBackEndService and lets the user execute some commands on it.

Now, let's say that MyView signs up for an event on MyBackEndService in order to display state changes in the UI. This will create a strong reference in MyBackEndService to MyView. If the user closes the MyView window, it should dispose itself and be garbage collected later. Suppose MyView doesn't detach from the event on MyBackEndService, the reference from MyBackEndService to MyView would still exists. This would make MyView never to be garbage collected until MyBackEndService is disposed! How to solve this problem? Who will save us? Weak Events!

Basically, the difference between a regular event and an weak event is that the former keeps strong references to the event handler delegates, while the latter uses WeakReference. This solves the problem that I've described beautifully.

WPF uses WeakEvent Patterns for most of its event system. As I said, I didn't want to make TrackableValue dependable on WPF. Fortunately, there is an excellent alternative. I used Weak Events from Daniel Grunwald. I strongly suggest reading his article if you are interested in the subject. I wouldn't be able to explain it better, so I won't try.

Conclusion

Although the code that I'm sharing here seems pretty neat to me, I have some questions about it. For example, should it really be a class, or should it be a struct instead? Also, is there a better way to control the sender parameter in the event handler (instead of using the Owner property)? What about a PreviewValueChanged event where some user code would be able to validate the change and, possibly, cancel it? If you have an opinion on these questions, I would be very happy to hear it.

Finally, here is the download link for the source code (v1.1 @ 2009.08.03). The zip file contains the WeakEvents source and, as usual, a demo project.

Have a nice day!

Updated at 2008.03.03: Source code link updated.


Dynamic Lambda Expressions Using An Isolated AppDomain
2008-11-27T20:35:00.008-02:00
Background

In August, I submitted a post about Lambda Expressions and Differentiation. There, I  told shortly about a dynamic compiler for lambda expressions using CodeDom. I've enhanced the compiler so that it now deserves a post of its own.

Introduction

I'll start with the original problem that I needed to solve. I wanted to make a console application that would accept lambda expressions wrote by the user, transform it in another lambda expression and show it. After some research, let me list some of the stuff that can or can't be done with .NET:One can write a lambda expressions to a easily readable string (that is similar to C# code, but it's not compilable);
One can dynamically create lambda expressions using static functions of the Expression class;
One cannot create (at least not directly) lambda expressions from a string containing valid C# code;
One can dynamically compile code using CodeDOM.

Due to feature #1, it's effortless to show the transformed lambda expressions to the user. Feature #2 is essential to the transformations that I intended to do, which was finding derivatives (please, read the original post if you're interested).

A major problem came along when I realized the fact #3. So, the source lambda expressions in the first versions of my little application were hard coded. After I finished the derivation part, I began searching for options to generate lambda expressions from user inputted strings.

One alternative was to make a parser (or find on the web) that put together an expression using feature #2. The other alternative is to let the framework do it by dynamically compiling the code (feature #4). Since the second is promptly available and is much more reliable than any parser I could code, that's what I chose to use.

Dynamic Compilation

Before I continue, let me state that I'll not get into the details about using CodeDOM. Here is a much more detailed explanation. Also, although you can compile VB.NET too, I'll talk only about C# here, sorry.

Basically, it works like this. You need the full text of one or more source code files (including usings, namespaces, classes, and so forth). Then, through a CodeDomProvider, you can compile your code to an Assembly. The assembly can exist only in memory or it could be generated directly to a file. You can access and execute what is in the assembly by reflection.

For my particular case, where I wanted to generate lambda expressions from a string, CodeDOM is too generic. So, I made a wrapper around it that compiles a lambda expression code string by filling the template below. Then, using reflection I executed DynamicFunc and forwarded the returned LambdaExpression.
using $Using0$;
using $Using1$;
//...
using $UsingN$;
namespace JpLabs.DynamicCode.DynamicAssembly
{
    public static class DynamicClass
    {
        public static LambdaExpression DynamicFunc() {
            return (Expression<$DelegateType$>)( $LambdaExpresion$ );
        }
    }
}


This solution solved my problem, but it has some issues. First, it's slow. Actually, it is not nearly as fast as it could be. One could say that I'm using an elephant gun to kill a fox. It's possible to modify it to compile several expressions at once, but that was not the case. I needed to compiled expressions one by one as the user inputs it.

The bigger issue is that, each time an expression is compiled, an assembly is generated. Unlike object instances, the memory of loaded types and assemblies can't be release. In fact, there's one way to do that, it's by unloading the entire application domain.

Cross AppDomain Dynamic Compilation

One application might use multiple domains for several reasons. But I believe nearly that all of them have to deal with one typical problem. Code in one domain cannot access data in other domain. So, they have to communicate in alternative ways. The two ways that are most straightforward are serialization and remoting.

The data I wanted to cross between domains are lambda expresssions. Unfortunately, a LambdaExpression is not Serializable. And remoting is not an option too. I need to be able to unload the compiler domain, and by doing this, all remoting objects would be released and the proxies would cease to work.

I thought I had hit a impassable wall. But I found this article with a solution. It also uses multiple domains to dynamically compile code. If you read to this point, you will probably like to take a look there too.

The idea somewhat crazy. .NET reflection allows you to access the IL of a method. One can pack it into a Serializable class and send it to another domain. There, the method would be reconstructed using the IL. This can be accomplished with the semi-magic DynamicMethod class. To let you understand its wonderful features, I'll quote MSDN remarks about it:
You can use the DynamicMethod class to generate and execute a method at run time, without having to generate a dynamic assembly and a dynamic type to contain the method. The executable code created by the just-in-time (JIT) compiler is reclaimed when the DynamicMethod object is reclaimed. Dynamic methods are the most efficient way to generate and execute small amounts of code.

The input for a DynamicMethod is exactly what we have: the IL that came from the compiled code on the other domain. The output is a delegate to a newly created (and releasable) function. Since the function returns our so desired lambda expression, we just need to call it.

Now, let me summarize the steps involved in this lambda expression compiler:Create an AppDomain;
Instantiate a compiler (actually, a wrapper around CodeDOM) on the new AppDomain;
Compile the input string into an in-memory assembly using the template I've shown above;
Unload the compiler domain (or keep it for reuse and unload it later);
Extract the IL of the DynamicFunc method (please, refer to the template), and send it to the original domain;
Employ a DynamicMethod to get a delegate from the compiled IL;
Call the delegate and it shall return a LambdaExpression.

Conclusion

Here's an reliable way to dynamically compile code without requiring always more memory. To be honest, I did some improvements that I didn't mention above. To mention, I tried to make the compiler generic enough so that it could compile other stuff than lambda expression. I think you can simply extend the compiler to achieve that. Please, tell me if anyone of you get to the point of doing that. I'll even let you in a short future-possible-enhancements-list:Extend compiler to compile other stuff than lambda expressions;
Improve the compiler so that it can compile a batch of code strips.

And, finally, the source code download link: here. Of course, this source includes a test project for the compiler. And, of course, the test project is my Differentiator.

Thanks!


LINQ Source Code (or LINQ for .NET 2.0)
2008-09-11T16:27:00.016-03:00
Background

Recently, I had to make some enhancements in a large stabilised asp.net solution. As usual, I began testing some code in a new project as a small proof of concept. After I was done, I realized that the old code was made using VS 2005 (i.e., over .NET Framework 2.0) and my test code used a lot of LINQ (i.e., over .NET Framework 3.5). Since upgrading the solution was not allowed, my only option was to remove Linq from my code. Or maybe not.

Using VS 2008, it's possible to use almost all the features of C# 3.0 even in a project targetting the 2.0 framework. The exception is Linq, because it inherently references extension functions that are only present in the 3.5 framework. Since I needed only Where() and Select(), I simply coded them and it worked just fine. I saw it was possible to write code for all the Linq to Objects extension methods. Better yet, I could aquire it from a more trustful source.

Linq to Objects Source Code

It's publicly known that a good part of the .NET Framework source code is available for anyone to download. Unfortunately, the code for Linq to Objects was not released yet. The alternative was to disassemble the assembly using .NET Reflector (I hope MS will not sue me for this). Anyway, as the code was not genuinely authored by me, I took special care of not using my namespace (JpLabs) in it.

One thing that Reflector don't do is to simplify functions that use yield return. Instead, it shows what is actually compiled, i.e., one compiler-generated iterator class for each function. These classes use state machines to control the iteretion. If you want to see how such a class looks like, try using Reflector. If you're not familiarised to yield return, you might want to check this arcticle. I decompiled all the iterator functions in Linq to Objects manually.

To make this post short, I would like to say that, in addition to the existent Linq extension functions, I've included some of my own, like ForEach(Action action).

Download

Download the full source here.

[Update @2008.09.25]
I've just found out that there is a project with the exactly the same purpose as the one in this post. And the authors are much more reliable than me. Check out the LINQBridge from the Albahari brothers. As mine, their code is completely open and free for any use.

Also, I would like to suggest the excellent free book on Threading in C# on their web page. It's definitely the most complete and most readable source on this subject that I know of.


Lambda Expressions and Differentiation (continued)
2008-08-14T16:50:00.009-03:00
Today, I made a presentation about the first article that I've posted on this blog. Since then I did some improvements on the code.

The one extra feature that I implemented was a parser for lambda expressions. One of the first things that I did when I started fiddling with lambda expression was trying to generate one from a string. To my disappointment, there's not such a feature included in the .NET Framework. So, I coded a humble one using CodeDom.

I'm sharing the last source code that I have. You're free to play with it with one condition. You have to share enhancements with me. :P

Also, I would like to let a challenge here. If you want to learn Lambda Expressions and don't have an idea to implement, try making code to simplify a polynomial function.

Finally, as always, please, let your comment here.


WebSphere MQ API (or MQAL)
2008-08-08T09:51:00.006-03:00
WebSphere MQ (or simply "MQ") is a messaging system made by IBM. If you've never used any message-oriented middleware, it might be a good idea to take a look at wikipedia first. Anyway, the basic problems that I had to solve in the component I'm posting are not specific to MOM systems. Thus, I hope you can extract some value from it in any case.

So, let me introduce you to the problem. About one year ago, I had to develop an application that extensively uses MQ to exchange messages with other applications. This technology was kind of new to me and I had to learn the basics. To oversimplify, the idea is to connect to a queue manager, then connect to a queue, and finally put (or get) a message to (or from) the queue. The way that this process is made using the MQ API for .NET seemed very awkward to me. Then, I decided to write a component to encapsulate it and provide a simpler interface.

Fisrt, I've tried to find a design pattern to fit the component in and I choosed to follow the way that database connections are handled in the .NET framework. Then, I've started to make some analogies. To start, a queue manager connection could work pretty much like a database connection. Both of them have parameters like server, port, user name, password, and database name / queue manager name. To summarize, here are some of the analogies I've made:
- A queue manager is like a database;
- A queue manager connection is like a database connection;
- A queue manager connection parameters is like a database connection string;
- A queue is like a table (or a view);
- A queue connection is like an database command;
- A message is like a table row;
- A put message action is like an execute insert command;
- A get message action is like an execute select and delete command;
- A browse message action is like an execute select command.

Ok, you must have gotten the general idea. I won't get in much detail here, but let me list some of the problems I had to solve in order to implement this:
- Translating a queue connection string into queue manager connection parameters. I solved this by coding a connection string parser using regular expressions;
- Implement queue transactions. This wasn't a requisite at the time, but MQ provides transactions and I felt like I had to offer a way to use them;
- Inherently use a queue connection pool under the covers. The tests that I've made using the MQ API showed that it doesn't had a connection pool. I would had to either make the user of my component keep connections for longer (in order to avoid opening and closing them too frequently) or code a connection pool and let the user enjoy all its benefits. This was pretty interesting to do and probably deserves a post of itself. For now I'll let you with the source code.

By the way, I didn't touched this code for awhile now and I see that I've let some items on a TODO list, so I won't say it's a final version. Anyway, it's running in a production environment somewhere for months and it have not failed once.

Feel free to download the source code here.


Microsoft Parallel Extensions CTP released
2008-06-03T17:58:00.006-03:00
In the past years, evolution of computing is turning to the grounds of high level concurrecy. Processors are becoming faster by being multi-core, and not by having higher frequencies. The time is coming so that even simple programs will be concurrent too. Parallel programs are harder to code, harder to test and can cause pseudo-random bugs that can drive programmers crazy. The best solution for this, as to many problems, is a dependable library and/or language support.

There are several types of problems that require specific solutions. Making a responsive UI, for instance. How many application have you seen that simply stay frozen while it's processing something? I've made some of them myself. And the programmer cannot be blamed, most projects don't have the time (and money) to invest on a responsive UI. The WPF framework solves this in a very neat way. Its interface engine is fully concurrent. But that's a subject for other post.

Yesterday, Microsoft released another CTP of its Parallel Extensions (PFX) library. You may get the details at their blog or at MSDN. In case you're not familiar with it, I'll give a short brief here. Basically, the library's comprehended by three parts:

- Task Parallel Library
Provides some imperative ways to express parallelism. The simpler one is the Invoke method. It receives a set of actions in the form of delegates and executes them concurrently, returning after all the actions have completed. If you have a loop in which iterations may run in parallel, you can use the For or the ForEach methods. When the problem requires a little more elaboration, there's the Task system. It's a set of classes that works somewhat like what the ThreadPool class does, but with much greater flexibility and control.

- PLINQ
It's a implementation of LINQ that allows anyone to easily make declarative parallel queries. All it's needed is to transform the IEnumerable<t> into a IParallelEnumerable<t> using the AsParallel() extention method. It's not optimized yet, but it certainly can boost many applications with almost no code change.

- Coordination Data Structures
It has a group of synchronization classes that enhance those that currently exists in the framework. Also, there are some thread-safe collection classes. Coincidently, they've created a ConcurrentQueue class that serves the same purpose as the one I've posted!

A detail that you can't avoid noticing in this framework is the extensive use of delegates. LINQ itself is based on delegates being passed to other functions. This type of construct is called a higher-order function. Another construct that is present in PFX is the continuation. Also, there are recurrent remarks in its documentation about side-effects and why they have to be avoided. All these characteristics are usually associated with functional langagues. No surprise some of the samples are in F#.

Functional programming has lived almost only at the academic world, but its future looks different. It holds a lot of interesting and useful concepts. The disturbing part is that very few programmers really know these concepts. Do you know any skilled/experienced/old programmer that can't understand OO programming? Well, can you really understand functional programming? I don't. At least not yet.


A C# BlockingQueue
2008-04-25T17:04:00.023-03:00
Ok, things didn't go as I planned, so I'll stop promising and start delivering. This time I'll share with you a code I've used before.

The problem is quite simple and is not new: exchange messages between threads. One possible solution is message queuing, but it's too bulky to use inside a single process. The simplest way to make threads communicate is through shared variables, but it's obviously too simple.

So, what kind of feature I was looking for? To begin with, it needed to be fast and light. Secondly, it had to work in a fire-and-forget fashion, i.e., I wanted a thread to be able to send a message (or task, job, whatever) to another thread and go on. Obviously, in order to such a thing to function, it would have to use some sort of internal buffer. So, it looked like what I needed was a thread-safe FIFO queue.

All right, I could simply use a Queue<t> and wrap its calls around a lock statement. Or, even easier, use a non-generic Queue and transform if by calling the Queue.Synchronized() method. In both ways, we'd get a simply synchronized thread-safe queue.

Is that all? Of course not. Suppose we have a producer-consumer problem to solve. Usually, when the buffer is empty, the consumer is supposed to idly wait for an new item. How could we accomplish that using a synched queue? If we try to dequeue from an empty Queue, a exception is thrown.  We could check the item count before dequeueing, but there would be two steps involved and it wouldn't be thread-safe even if the Queue instance is synched. Another possibility is catching the queue-is-empty exception, which is a very lousy trick. Anyway, we wouldn't be able to avoid writing synchronization code and this is not what I wanted.

So, considering that you are introduced to the problem, let me show the wish list that I gathered in the form of an interface:
public interface IBlockingQueue
{
/// <summary>Max number of item in the queue (-1 means there's no limit)</summary>
int MaxQueueDepth { get; }

/// <summary>Returns false when there's something wrong, e.g., queue full</summary>
bool TryEnqueue(object item);

/// <summary>Returns false when there's no message to read</summary>
bool TryDequeue(out object item);

/// <summary>Returns false when a timeout or an abort happens</summary>
bool WaitEnqueue(object item, int millisecondsTimeout, FuncGetVolatileFlag ExternalAbortSignaled);

/// <summary>Returns false when a timeout or an abort happens</summary>
bool WaitDequeue(out object item, int millisecondsTimeout, FuncGetVolatileFlag ExternalAbortSignaled);
}

public delegate bool FuncGetVolatileFlag();

TryDequeue() and TryEnqueue are two simply synchronized methods to access the queue. The other two (WaitDequeue() and WaitEnqueue()) are much more interesting. The first one will try to dequeue and, if the queue is empty, will wait for a new item. The latter will try to enqueue and, if the queue is full, will wait until there's room for the new item.

But that's still not enough. A common problem in a multithreaded application is when you need to close it and there are some threads still running. There's several possible approaches on how to close them. The safer choice might be to wait for every thread to end, but it might take too long. One alternative is to terminate everyone, but it's very risky. I think the best pick is to ask somehow the threads to stop and wait until they do so. With my blocking queue, you can choose between any of the 3.

Let me explain how. As you can see, I've put two addicional parameters. Both have the same purpose: cancel the action and return the code execution to the client code. Whilst the fisrt is obviously a timeout, one might wonder what that other parameter is. It's a delegate of a type that accepts functions that like this:bool func(). Its name is ExternalAbortSignaled and its type name is FuncGetVolatileFlag, so I hope it's not hard to recognize how it's used.

In order to eliminate any doubts, I'll describe it. The idea is that, while waiting, the ExternalAbortSignaled delegate will be called in short intervals. If it returns true at any time, the waiting ends and no item is enqueued/dequeued. Also the function returns false. In that way, it's possible to provide the queue an anonymous method that merely returns the value of a boolean flag. Whenever the flag is set to true, the wait function will return promptly and the application as a whole will be able to respond faster.

As you may know, java has a built-in BlockingQueue already. Sadly, java has direct support for delegates. So, I think it would be hard to code a functionality like an abort flag in an elegant way.

Now, if you want to know what is the best way to implement the waiting synchronization code, I'll let the explanation to Joseph, which is much more eloquent than me. It might be a good idea to take a look at my source code too. The technique that I've chosen is through Wait and Pulse.

Some ideas for future enhancements that I've thought of are:
- TryPeek() and WaitPeek()
- Transactions

Source code download link at google code 

[Updated at 2008-04-28]


Hello, Again
2008-02-19T11:36:00.001-03:00
I was on vacations until recently and I'm quite busy now, so it might take longer than I expected to write a new article. I guess it's a good thing to publish short posts once in a while.

Anyway, what I wanted to tell you is that a group that I'm part of is organizing an event here in Porto Alegre. I'll make a presentation about LINQ to Relational Data. Very much of it is 'inspired' in material from Mike Pizzo, but I'll show some stuff of my own.

Since I'm studing LINQ, I'm trying to prepare an interesting article about it. If you can't wait and want to get a deep dive into it, I can recomend a very exhaustive series of posts on LINQ to LDAP by Bart De Smet's.

See ya!


Lambda Expressions and Higher-Order Functions
2008-01-07T15:30:00.006-02:00
[This article has a continuation here]

First, I need to ask you not to be scare of the title I've chosen. Less than two months ago, I barely knew what the first name was and have never heard of the latter.

As you might know, the brand-new C# 3.0 has support to lambda expressions. You can imagine it as a less wordy way to describe anonymous delegates. A major difference between them is that a lambda expression doesn't necessarily needs to be compiled. In fact, it's just a representation of a function. In addition to being compiled, they can be serialized (if you want to pass it on), analyzed and even dynamically created at run-time. I'll be using the last two here.

Now, let me introduce the computational (as well as mathematical) concept of a higher-order funcion (aka operators in math). It's nothing more than a function that receives another function as a parameter and/or returns a function.

Why would a function receive another function as a parameter? If you can't imagine a good answer or are just not convinced there's one, I suggest that you read this very didactic article (this is also good). Anyway, you can implement such functions in c# 2.0 without problems. What I want to examine are functions that returns another functions. If you are not using c# 3.0 (or a functional language), you might find some difficulties. Basically, you would be limited to returning one of a set of pre-declared functions. You could try using closures or dynamic code generation to add some versatility, but it probably would be rather cumbersome.

You may be asking why would anyone need such higher-order function. I will let this question open and focus on how to implement it. I was searching for a good example and can't imagine a better one than the differentiation, i.e., the mathematical process of obtaining the derivative of a function.

So, I'll build a functional that receives a f(x) and return a f'(x). The declaration of this in c# would be like that:
static public Expression<Func<double,double>> Differentiate(this Expression<Func<double,double>> f)
{
//I will find the derivative of f with respect to the variable x (it's the 1st and sole parameter)
ParameterExpression x = f.Parameters[0];

//Diff is the function that actually does the differentiation
Expression derivative = Diff(f.Body, x);

//Build a new lambda expression with the derivative expression body
return Expression.Lambda<Func<double,double>>(derivative, f.Parameters);
}
*Func<double,double> is a delegate type for functions that receives a single double parameter and returns a double. Expression<Func<double,double>> is the type of a lambda expression that represents a Func<double,double> function.

As you can see I used a Diff function that actually calculates the derivative. How does it work? It needs to analyze the original expression and create a new expression that represents its derivative. Luckily enough, this calculation can be done recursively using a set a rules. Let's start with the sum rule: namely, derivatives of sums are equal to the sum of derivatives. Coding that in c# 3.0 is easier than you one might thought. Look:
static Expression Diff(Expression exp, ParameterExpression dx)
{
switch (exp.NodeType) {
case ExpressionType.Add: {
BinaryExpression f = (BinaryExpression)exp;
//Sum rule: [g(x) + h(x)]' == g'(x) + h'(x).
return Expression.Add( //Create an add expression
   Diff(f.Left, dx),  //g'(x) +
   Diff(f.Right, dx)  //h'(x)
);
}
/* other cases omitted ... */
}
}

In similar ways, we can implement the other recursive rules (such as the product rule, the quotient rule, the power rule, etc). However, any recursion need a stop condition. We'll have two stop conditions: for constants, which derivative is zero; and for the variable x, which derivative is one. The code looks like this:
case ExpressionType.Parameter: {
//be aware that f(x) = y is a constant with respect to x
return (exp == dx) ? Expression.Constant(1d) : Expression.Constant(0d);
}
case ExpressionType.Constant: {
return Expression.Constant(0d);
}

There's also the chain rule, that is quite trickier to implement. I did it with very little code, though. As you're not faint of heart, take a look at my full source.

And that's all for the day, folks. Please, comment.


Hello World
2008-01-04T12:56:00.000-02:00
Hi,

My name is João Paulo Bochi and I'm a senior programmer in Brazil. Welcome to my knowledge sharing blog.

I'll post mainly C# related stuff with tips and tricks that I learned. I plan to include topics about C#, LINQ, ASP.NET, AJAX, Concurrent Programming and some others I am (or might get) interested with.

Please, pay a visit soon.