The idea of the thread throttling class

One particular problem in programming is when you have a thread which only serves the purpose of doing some trivial repetetive work in a timely manner. Most often these threads are implemented as:

while (isRunning) {
    doSimpleJob();
    sleep(Seconds(1));
}

In a perfect world all threads wait for signals rather than implementing this kind of busy-wait polling but when you need something simple like above I believe something as simple as busy-wait could be considered good enough in most cases.

The purpose of the sleep in the case stated above is to ensure that if doSimpleJob() completes in very few ticks we won’t end up unintentionally starving other threads while doing our trivial low priority task.

The downside to this is that if it is difficult to determine how long doSimpleJob() takes to complete, i.e. if it’s waiting for I/O, it will also be difficult to set a good timeout for sleep. If it takes five seconds to complete the job there is hardly no need to sleep for another second, and maybe you even want to run it again as fast as you can if it took a long time to complete. The solution I’m proposing is to implement a type that can determine how long we should sleep, if at all. Some pseudocode:

class Throttle {
public:
    Throttle(Milliseconds timeout) :
        m_start(Time::now()),
        m_timeout(timeout) { }

    void sleep() {
        Milliseconds duration(Time::now() - m_start);
        if (duration < m_timeout) {
            sleep(m_timeout - duration);
       }
    }

private:
    Time m_start;
    Milliseconds m_timeout;
}

Using this implementation the example mentioned above would become:

while (isRunning) {
    Throttle throttle(Seconds(1));
    doSimpleJob();
    throttle.sleep();
}

Now if doSimpleJob() takes less than a second one loop iteration would not take up more than one second and if it takes more than a second it will be instantly restarted.

One danger to this is that if doSimpleJob() is computation bound rather than I/O bound it might end up starving threads if it takes more time than the throttle timeout so it has to be applied with care.

For the sake of the argument lets assume that the implementation of sleep() being used actually sleeps for the whole duration of the specified time, no more no less.