改代码前

This commit is contained in:
2026-08-26 10:55:29 +08:00
parent 65799cfc2a
commit 5ecb12b8f7
59 changed files with 4224 additions and 2458 deletions
+31
View File
@@ -0,0 +1,31 @@
cmake_minimum_required(VERSION 2.8)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -g3 -Wall -Werror -Wno-sign-compare -O3")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c99 -g3 -Wall -Werror -Wno-sign-compare -O3")
add_executable(test_basic.testbin
src/test/test_basic.cc)
add_executable(test_benchmark.testbin
src/test/test_benchmark.cc)
enable_testing()
add_test(test_basic bin/test_basic.testbin)
# CMake test support is a total shitshow. The test targets don't have
# a dependency on the test binary, and it's in fact impossible to add
# any dependencies at all for a test target. This means that "make
# test" will never do the right thing, but just run some random
# previously compiled versions.
#
# All the workarounds suck. This one seems to suck the least; add a
# test that builds the other tests, and then add this non-standard
# test-only sequencing dependency to force that test to run first.
add_test(build_test_code "${CMAKE_COMMAND}" --build ${CMAKE_BINARY_DIR} --target all)
set_tests_properties(test_basic PROPERTIES DEPENDS build_test_code)
+22
View File
@@ -0,0 +1,22 @@
MIT License
Copyright (c) 2016 Juho Snellman
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+179
View File
@@ -0,0 +1,179 @@
*** Ratas - A hierarchical timer wheel
A timer queue which allows events to be scheduled for execution
at some later point. Reasons you might want to use this implementation
instead of some other are:
- A single-file C++11 implementation with no external dependencies.
- Optimized for high occupancy rates, on the assumption that the
utilization of the timer queue is proportional to the utilization
of the system as a whole. When a tradeoff needs to be made
between efficiency of one operation at a low occupancy rate and
another operation at a high rate, we choose the latter.
- Tries to minimize the cost of event rescheduling or cancelation,
on the assumption that a large percentage of events will never
be triggered. The implementation avoids unnecessary work when an
event is rescheduled, and provides a way for the user specify a
range of acceptable execution times instead of just an exact one.
- Facility for limiting the number of events to execute on a
single invocation, to allow fine grained interleaving of timer
processing and application logic.
- An interface that at least the author finds convenient.
The exact implementation strategy is a hierarchical timer
wheel. A timer wheel is effectively a ring buffer of linked lists
of events, and a pointer to the ring buffer. As the time advances,
the pointer moves forward, and any events in the ring buffer slots
that the pointer passed will get executed.
A hierarchical timer wheel layers multiple timer wheels running at
different resolutions on top of each other. When an event is
scheduled so far in the future than it does not fit the innermost
(core) wheel, it instead gets scheduled on one of the outer
wheels. On each rotation of the inner wheel, one slot's worth of
events are promoted from the second wheel to the core. On each
rotation of the second wheel, one slot's worth of events is
promoted from the third wheel to the second, and so on.
*** Usage
The basic usage is to create a single =TimerWheel= object and
multiple =TimerEvent= or =MemberTimerEvent= objects. The events are
scheduled for execution using =TimerWheel::schedule()= or
=TimerWheel::schedule_in_range()=, or unscheduled using the event's
=cancel()= method. The callbacks of the =TimerEvent= objects will
get triggered during call =TimerWheel::advance()=, once the time
advances far enough.
**** TimerEventInterface
An abstract class representing an event that can be scheduled to
happen at some later time.
***** =TimerEventInterface::~TimerEventInterface()=
TimerEvents are automatically canceled on destruction.
***** =TimerEventInterface::cancel()=
Unschedule this event. It's safe to cancel an event that is inactive.
***** =TimerEventInterface::active()=
Return true iff the event is currently scheduled for execution.
***** =TimerEventInterface::scheduled_at()=
Return the absolute tick this event is scheduled to be executed on.
**** =TimerEvent<CBType>=
An event that takes the callback (of type =CBType=) to execute as
a constructor parameter.
**** =MemberTimerEvent<T, MFun>=
An event that's specialized with a (static) member function of class =T=,
and a dynamic instance of =T=. Event execution causes an invocation of the
member function on the instance.
**** =TimerWheel=
A =TimerWheel= is the entity that =TimerEvents= can be scheduled on
for execution (with =schedule()= or =schedule_in_range()=), and will
eventually be executed once the time advances far enough with the
=advance()= method.
***** =TimerWheel::advance(Tick delta, size_t max_execute = ..., int level = 0)=
Advance the TimerWheel by the specified number of ticks (=delta=), and execute
any events scheduled for execution at or before that time. The
number of events executed can be restricted using the =max_execute=
parameter. If that limit is reached, the function will return false,
and the excess events will be processed on a subsequent call.
- It is safe to cancel or schedule events from within event callbacks.
- During the execution of the callback the observable event tick will
be the tick it was scheduled to run on; not the tick the clock will
be advanced to.
- Events will happen in order; all events scheduled for tick X will
be executed before any event scheduled for tick X+1.
Delta should be non-0. The only exception is if the previous
call to =advance()= returned false.
=advance()= should not be called from an event callback.
The =level= parameter is used to trigger timer advances on different
levels of the hierarchy. It will generally not be useful to pass in
any value other than the default 0.
***** =TimerWheel::schedule(TimerEventInterface* event, Tick delta)=
Schedule the event to be executed =delta= ticks from the current time.
The delta must be non-0.
***** =TimerWheel::schedule_in_range(TimerEventInterface* event, Tick start, Tick end)=
Schedule the event to happen at some time between start and end
ticks from the current time. The actual time will be determined
by the =TimerWheel= to minimize rescheduling and promotion overhead.
Both =start= and =end= must be non-0, and =end= must be greater than
=start=.
***** =TimerWheel::now()=
Return the current tick value. Note that if the time increases
by multiple ticks during a single call to advance(), during the
execution of the event callback now() will return the tick that
the event was scheduled to run on.
***** =TimerWheel::ticks_to_next_event(Tick max = ..., int level = 0)=
Return the number of ticks remaining until the next event will get
executed. If the max parameter is passed, that will be the maximum
tick value that gets returned. The max parameter's value will also
be returned if no events have been scheduled.
Will return 0 if the wheel still has unprocessed events from the
previous call to advance().
The =level= parameter is used to trigger timer advances on different
levels of the hierarchy. It will generally not be useful to pass in
any value other than the default 0.
*** Examples
#+BEGIN_SRC
typedef std::function<void()> Callback;
TimerWheel timers;
int count = 0;
TimerEvent<Callback> timer([&count] () { ++count; });
timers.schedule(&timer, 5);
timers.advance(4);
assert(count == 0);
timers.advance(1);
assert(count == 1);
timers.schedule(&timer, 5);
timer.cancel();
timers.advance(4);
assert(count == 1);
#+END_SRC
To tie events to specific member functions of an object instead of
a callback function, use MemberTimerEvent instead of TimerEvent.
For example:
#+BEGIN_SRC
class Test {
public:
Test() : inc_timer_(this) {
}
void start(TimerWheel* timers) {
timers->schedule(&inc_timer_, 10);
}
void on_inc() {
count_++;
}
int count() { return count_; }
private:
MemberTimerEvent<Test, &Test::on_inc> inc_timer_;
int count_ = 0;
};
#+END_SRC
+435
View File
@@ -0,0 +1,435 @@
// -*- mode: c++; c-basic-offset: 4 indent-tabs-mode: nil -*- */
//
// Copyright 2016 Juho Snellman, released under a MIT license (see
// LICENSE).
#include <algorithm>
#include <functional>
#include <vector>
#include "../timer-wheel.h"
#define TEST(fun) \
do { \
if (fun()) { \
printf("[OK] %s\n", #fun); \
} else { \
ok = false; \
printf("[FAILED] %s\n", #fun); \
} \
} while (0)
#define EXPECT(expr) \
do { \
if (!(expr)) { \
printf("%s:%d: Expect failed: %s\n", \
__FILE__, __LINE__, #expr); \
return false; \
} \
} while (0)
#define EXPECT_INTEQ(actual, expect) \
do { \
if (expect != actual) { \
printf("%s:%d: Expect failed, wanted %ld" \
" got %ld\n", \
__FILE__, __LINE__, \
(long) expect, (long) actual); \
return false; \
} \
} while (0)
bool test_single_timer_no_hierarchy() {
typedef std::function<void()> Callback;
TimerWheel timers;
int count = 0;
TimerEvent<Callback> timer([&count] () { ++count; });
// Unscheduled timer does nothing.
timers.advance(10);
EXPECT_INTEQ(count, 0);
EXPECT(!timer.active());
// Schedule timer, should trigger at right time.
timers.schedule(&timer, 5);
EXPECT(timer.active());
timers.advance(5);
EXPECT_INTEQ(count, 1);
// Only trigger once, not repeatedly (even if wheel wraps
// around).
timers.advance(256);
EXPECT_INTEQ(count, 1);
// ... unless, of course, the timer gets scheduled again.
timers.schedule(&timer, 5);
timers.advance(5);
EXPECT_INTEQ(count, 2);
// Canceled timers don't run.
timers.schedule(&timer, 5);
timer.cancel();
EXPECT(!timer.active());
timers.advance(10);
EXPECT_INTEQ(count, 2);
// Test wraparound
timers.advance(250);
timers.schedule(&timer, 5);
timers.advance(10);
EXPECT_INTEQ(count, 3);
// Timers that are scheduled multiple times only run at the last
// scheduled tick.
timers.schedule(&timer, 5);
timers.schedule(&timer, 10);
timers.advance(5);
EXPECT_INTEQ(count, 3);
timers.advance(5);
EXPECT_INTEQ(count, 4);
// Timer can safely be canceled multiple times.
timers.schedule(&timer, 5);
timer.cancel();
timer.cancel();
EXPECT(!timer.active());
timers.advance(10);
EXPECT_INTEQ(count, 4);
{
TimerEvent<Callback> timer2([&count] () { ++count; });
timers.schedule(&timer2, 5);
}
timers.advance(10);
EXPECT_INTEQ(count, 4);
return true;
}
bool test_single_timer_hierarchy() {
typedef std::function<void()> Callback;
TimerWheel timers;
int count = 0;
TimerEvent<Callback> timer([&count] () { ++count; });
EXPECT_INTEQ(count, 0);
// Schedule timer one layer up (make sure timer ends up in slot 0 once
// promoted to the innermost wheel, since that's a special case).
timers.schedule(&timer, 256);
timers.advance(255);
EXPECT_INTEQ(count, 0);
timers.advance(1);
EXPECT_INTEQ(count, 1);
// Then schedule one that ends up in some other slot
timers.schedule(&timer, 257);
timers.advance(256);
EXPECT_INTEQ(count, 1);
timers.advance(1);
EXPECT_INTEQ(count, 2);
// Schedule multiple rotations ahead in time, to slot 0.
timers.schedule(&timer, 256*4 - 1);
timers.advance(256*4 - 2);
EXPECT_INTEQ(count, 2);
timers.advance(1);
EXPECT_INTEQ(count, 3);
// Schedule multiple rotations ahead in time, to non-0 slot. (Do this
// twice, once starting from slot 0, once starting from slot 5);
for (int i = 0; i < 2; ++i) {
timers.schedule(&timer, 256*4 + 5);
timers.advance(256*4 + 4);
EXPECT_INTEQ(count, 3 + i);
timers.advance(1);
EXPECT_INTEQ(count, 4 + i);
}
return true;
}
bool test_ticks_to_next_event() {
typedef std::function<void()> Callback;
TimerWheel timers;
TimerEvent<Callback> timer([] () { });
TimerEvent<Callback> timer2([] () { });
// No timers scheduled, return the max value.
EXPECT_INTEQ(timers.ticks_to_next_event(100), 100);
EXPECT_INTEQ(timers.ticks_to_next_event(),
std::numeric_limits<Tick>::max());
for (int i = 0; i < 10; ++i) {
// Just vanilla tests
timers.schedule(&timer, 1);
EXPECT_INTEQ(timers.ticks_to_next_event(100), 1);
timers.schedule(&timer, 20);
EXPECT_INTEQ(timers.ticks_to_next_event(100), 20);
// Check the the "max" parameters works.
timers.schedule(&timer, 150);
EXPECT_INTEQ(timers.ticks_to_next_event(100), 100);
// Check that a timer on the next layer can be found.
timers.schedule(&timer, 280);
EXPECT_INTEQ(timers.ticks_to_next_event(100), 100);
EXPECT_INTEQ(timers.ticks_to_next_event(1000), 280);
// Test having a timer on the next wheel (still remaining from
// the previous test), and another (earlier) timer on this
// wheel.
for (int i = 1; i < 256; ++i) {
timers.schedule(&timer2, i);
EXPECT_INTEQ(timers.ticks_to_next_event(1000), i);
}
timer.cancel();
timer2.cancel();
// And then run these same tests from a bunch of different
// wheel locations.
timers.advance(32);
}
// More thorough tests for cases where the next timer could be on
// either of two different wheels.
for (int i = 0; i < 20; ++i) {
timers.schedule(&timer, 270);
timers.advance(128);
EXPECT_INTEQ(timers.ticks_to_next_event(512), 270 - 128);
timers.schedule(&timer2, 250);
EXPECT_INTEQ(timers.ticks_to_next_event(512), 270 - 128);
timers.schedule(&timer2, 10);
EXPECT_INTEQ(timers.ticks_to_next_event(512), 10);
// Again, do this from a bunch of different locatoins.
timers.advance(32);
}
timer.cancel();
EXPECT_INTEQ(timers.ticks_to_next_event(),
std::numeric_limits<Tick>::max());
return true;
}
bool test_schedule_in_range() {
typedef std::function<void()> Callback;
TimerWheel timers;
TimerEvent<Callback> timer([] () { });
// No useful rounding possible.
timers.schedule_in_range(&timer, 281, 290);
EXPECT_INTEQ(timers.ticks_to_next_event(), 290);
// Pick a time aligned at slot boundary if possible.
timers.schedule_in_range(&timer, 256*4 - 1, 256*5 - 1);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*4);
timers.schedule_in_range(&timer, 256*4 + 1, 256*5);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*5);
// Event already in right range.
timers.schedule_in_range(&timer, 256*1, 256*10);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*5);
// Event canceled, but was previously scheduled in
// the right range. Should be ignored, and scheduled
// as normal to the end of the range.
timer.cancel();
timers.schedule_in_range(&timer, 256*1, 256*10);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*10);
// Make sure the decision on whether timer is in range or
// not is done based on absolute ticks, not relative ticks.
timers.advance(256*9);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*1);
timers.schedule_in_range(&timer, 256*9, 256*10);
EXPECT_INTEQ(timers.ticks_to_next_event(), 256*10);
// Try scheduling timers in random ranges.
for (int i = 0; i < 10000; ++i) {
int len1 = rand() % 20;
int len2 = rand() % 20;
int r1 = rand() % (1 << len1);
int r2 = r1 + (1 + rand() % (1 << len2));
timers.schedule_in_range(&timer, r1, r2);
EXPECT(timers.ticks_to_next_event() >= r1);
EXPECT(timers.ticks_to_next_event() <= r2);
}
return true;
}
bool test_reschedule_from_timer() {
typedef std::function<void()> Callback;
TimerWheel timers;
int count = 0;
TimerEvent<Callback> timer([&count] () { ++count; });
// For every slot in the outermost wheel, try scheduling a timer from
// a timer handler 258 ticks in the future. Then reschedule it in 257
// ticks. It should never actually trigger.
for (int i = 0; i < 256; ++i) {
TimerEvent<Callback> rescheduler([&timers, &timer] () { timers.schedule(&timer, 258); });
timers.schedule(&rescheduler, 1);
timers.advance(257);
EXPECT_INTEQ(count, 0);
}
// But once we stop rescheduling the timer, it'll trigger as intended.
timers.advance(2);
EXPECT_INTEQ(count, 1);
return true;
}
bool test_single_timer_random() {
typedef std::function<void()> Callback;
TimerWheel timers;
int count = 0;
TimerEvent<Callback> timer([&count] () { ++count; });
for (int i = 0; i < 10000; ++i) {
int len = rand() % 20;
int r = 1 + rand() % ( 1 << len);
timers.schedule(&timer, r);
if (r > 1)
timers.advance(r - 1);
EXPECT_INTEQ(count, i);
timers.advance(1);
EXPECT_INTEQ(count, i + 1);
}
return true;
}
bool test_maxexec() {
typedef std::function<void()> Callback;
TimerWheel timers;
int count0 = 0;
int count1 = 0;
TimerEvent<Callback> timer0([&count0] () { ++count0; });
TimerEvent<Callback> timer1a([&count1] () { ++count1; });
TimerEvent<Callback> timer1b([&count1] () { ++count1; });
// Schedule 3 timers to happen at the same time (on 2 different
// wheels).
timers.schedule(&timer1a, 256);
timers.schedule(&timer1b, 256);
timers.advance(1);
timers.schedule(&timer0, 255);
timers.advance(254);
EXPECT_INTEQ(count0, 0);
EXPECT_INTEQ(count1, 0);
EXPECT_INTEQ(timers.ticks_to_next_event(), 1);
EXPECT_INTEQ(timers.now(), 255);
// Then run them one by one.
EXPECT(!timers.advance(1, 1));
EXPECT_INTEQ(count0, 0);
EXPECT_INTEQ(count1, 1);
EXPECT_INTEQ(timers.ticks_to_next_event(), 0);
// Note that time has already advanced.
EXPECT_INTEQ(timers.now(), 256);
EXPECT(!timers.advance(0, 1));
EXPECT_INTEQ(count0, 0);
EXPECT_INTEQ(count1, 2);
EXPECT(!timers.advance(0, 1));
EXPECT_INTEQ(count0, 1);
EXPECT_INTEQ(count1, 2);
// We have not finished the tick yet, since the last call exactly
// drained the queue. But the next call will finish the tick while
// doing no actual work.
EXPECT_INTEQ(timers.ticks_to_next_event(100), 0);
EXPECT(timers.advance(0, 1));
EXPECT_INTEQ(timers.ticks_to_next_event(100), 100);
// Test scheduling while wheel is in the middle of partial tick handling.
timers.schedule(&timer1a, 256);
timers.advance(1);
timers.schedule(&timer0, 255);
timers.advance(254);
EXPECT(!timers.advance(1, 1));
// Now in the middle of the tick.
std::vector<bool> done(false, 512);
std::vector<TimerEvent<Callback>*> events;
// Schedule 512 timers, each setting the matching bit in "done".
for (int i = 0; i < done.size(); ++i) {
auto event = new TimerEvent<Callback>([&done, i] () { done[i] = true; });
events.push_back(event);
timers.schedule(event, i + 1);
}
// Close the tick.
EXPECT(timers.advance(0, 100));
// Now check that all 512 timers were scheduled in the right location.
for (int i = 0; i < done.size(); ++i) {
EXPECT_INTEQ(std::count(done.begin(), done.end(), true), i);
EXPECT(!done[i]);
timers.advance(1);
EXPECT(done[i]);
}
return true;
}
class Test {
public:
Test()
: inc_timer_(this), reset_timer_(this) {
}
void start(TimerWheel* timers) {
timers->schedule(&inc_timer_, 10);
timers->schedule(&reset_timer_, 15);
}
void on_inc() {
count_++;
}
void on_reset() {
count_ = 0;
}
int count() { return count_; }
private:
MemberTimerEvent<Test, &Test::on_inc> inc_timer_;
MemberTimerEvent<Test, &Test::on_reset> reset_timer_;
int count_ = 0;
};
bool test_timeout_method() {
TimerWheel timers;
Test test;
test.start(&timers);
EXPECT_INTEQ(test.count(), 0);
timers.advance(10);
EXPECT_INTEQ(test.count(), 1);
timers.advance(5);
EXPECT_INTEQ(test.count(), 0);
return true;
}
int main(void) {
bool ok = true;
TEST(test_single_timer_no_hierarchy);
TEST(test_single_timer_hierarchy);
TEST(test_ticks_to_next_event);
TEST(test_schedule_in_range);
TEST(test_single_timer_random);
TEST(test_maxexec);
TEST(test_reschedule_from_timer);
TEST(test_timeout_method);
// Test canceling timer from within timer
return ok ? 0 : 1;
}
@@ -0,0 +1,265 @@
// -*- mode: c++; c-basic-offset: 4 indent-tabs-mode: nil -*-
//
// Copyright 2016 Juho Snellman, released under a MIT license (see
// LICENSE).
#include <algorithm>
#include <functional>
#include <string>
#include <sys/time.h>
#include <sys/resource.h>
#include "../timer-wheel.h"
static bool allow_schedule_in_range = true;
// Set to true to print a trace, to confirm that different timer
// implementations give the same results. (Or close enough results,
// if using non-deterministic features like schedule_in_range).
static bool print_trace = true;
static int pair_count = 5;
// The total number of response messages received on all units.
// Printed in the final output, useful as a poor man's output checksum.
static long total_rx_count = 0;
// Pretend we're using timer ticks of 20 microseconds. So 50000 ticks
// is one second.
static Tick time_ms = 50;
static Tick time_s = 1000*time_ms;
class Unit {
public:
Unit(TimerWheel* timers, int request_interval=1*time_s)
: timers_(timers),
idle_timer_(this),
close_timer_(this),
pace_timer_(this),
request_timer_(this),
request_deadline_timer_(this),
id_(id_counter_++),
request_interval_ticks_(request_interval) {
}
~Unit() {
if (print_trace) {
printf("delete %d, rx-count=%d\n", id_, rx_count_);
}
total_rx_count += rx_count_;
}
// Create a full work unit from the two halves.
void pair_with(Unit* other) {
other_ = other;
}
// Start the benchmark, acting either as the client or the server.
void start(bool server) {
unidle();
// Start shutdown of this work unit in 180s.
timers_->schedule(&close_timer_, 180*time_s);
if (!server) {
// Fire off the first server from the client.
on_request();
}
}
// Queue "count" messages to be transmitted.
void transmit(int count) {
tx_count_ += count;
deliver();
}
// Deliver as many response messages as we have quota for. Then
// start off a timer to refresh the quota.
void deliver() {
unidle();
int amount = std::min(pace_quota_, tx_count_);
pace_quota_ -= amount;
tx_count_ -= amount;
other_->receive(amount);
if (!pace_quota_) {
timers_->schedule(&pace_timer_, pace_interval_ticks_);
}
}
// Receive some number of response messages.
void receive(int count) {
unidle();
// Receive the first response to a given request. Move the
// deadline timer back in time (since this connection is now
// clearly active).
if (waiting_for_response_) {
timers_->schedule(&request_deadline_timer_,
pace_interval_ticks_ * RESPONSE_SIZE * 2);
waiting_for_response_ = false;
}
rx_count_++;
// We've received the full response. Stop the deadline timer,
// and start another timer that'll trigger the next request.
if (rx_count_ % RESPONSE_SIZE == 0) {
request_deadline_timer_.cancel();
timers_->schedule(&request_timer_, request_interval_ticks_);
}
}
// First time this timer gets executed, we put the object into a
// closing state where it'll start winding down work. Then we
// forcibly close it a bit later. We do it like this to remove any
// non-determinism between the execution order of the close timer
// and the pace timer.
void on_close() {
if (closing_) {
delete this;
} else {
closing_ = true;
timers_->schedule(&close_timer_, 10*time_s);
}
}
// Refresh transmit quota.
void on_pace() {
if (tx_count_) {
pace_quota_ = 1;
deliver();
}
}
// The endpoint has been idle for too long, kill it.
void on_idle() {
delete this;
}
// Send a new request (unless were draining traffic).
void on_request() {
if (!closing_) {
// Expect a response within this time.
timers_->schedule(&request_deadline_timer_,
pace_interval_ticks_ * RESPONSE_SIZE * 4);
waiting_for_response_ = true;
other_->transmit(RESPONSE_SIZE);
}
}
// We've done some work. Move the idle timer further into the future.
void unidle() {
if (allow_schedule_in_range) {
timers_->schedule_in_range(&idle_timer_, 60*time_s,
61*time_s);
} else {
timers_->schedule(&idle_timer_, 60*time_s);
}
}
// Something has gone wrong. Forcibly close down both sides.
void on_request_deadline() {
fprintf(stderr, "Request did not finish by deadline\n");
delete this;
delete other_;
}
private:
TimerWheel* timers_;
// This timer gets rescheduled far into the future at very frequent
// intervals.
MemberTimerEvent<Unit, &Unit::on_idle> idle_timer_;
// This timers gets scheduled twice, and executed twice.
MemberTimerEvent<Unit, &Unit::on_close> close_timer_;
// This gets scheduled very soon at frequent intervals, and is always
// executed.
MemberTimerEvent<Unit, &Unit::on_pace> pace_timer_;
// This gets scheduled about 150-200 times during the benchmark a
// medium duration from now, and is always executed
MemberTimerEvent<Unit, &Unit::on_request> request_timer_;
// This gets scheduled at a medium duration 150-200 times during a
// benchmark, but always gets canceled (not rescheduled).
MemberTimerEvent<Unit, &Unit::on_request_deadline> request_deadline_timer_;
static int id_counter_;
const static int RESPONSE_SIZE = 128;
int id_;
int tx_count_ = 0;
int rx_count_ = 0;
Unit* other_ = NULL;
int pace_quota_ = 1;
int pace_interval_ticks_ = 10;
int request_interval_ticks_;
bool closing_ = false;
bool waiting_for_response_ = false;
};
int Unit::id_counter_ = 0;
static void make_unit_pair(TimerWheel* timers, int request_interval) {
Unit* server = new Unit(timers);
Unit* client = new Unit(timers, request_interval);
server->pair_with(client);
client->pair_with(server);
server->start(true);
client->start(false);
}
bool bench() {
TimerWheel timers;
// Create the events evenly spread during this time range.
int create_period = 1*time_s;
double create_progress_per_iter = (double) pair_count / create_period * 2;
double current_progress = 0;
long int count = 0;
while (timers.now() < create_period) {
current_progress += (rand() * create_progress_per_iter) / RAND_MAX;
while (current_progress > 1) {
--current_progress;
make_unit_pair(&timers, 1*time_s + rand() % 100);
++count;
}
timers.advance(1);
}
fprintf(stderr, "%ld work units (%ld timers)\n",
count, count * 10);
while (timers.now() < 300*time_s) {
Tick t = timers.ticks_to_next_event(100*time_ms);
timers.advance(t);
}
return true;
}
int main(int argc, char** argv) {
if (char* s = getenv("BENCH_ALLOW_SCHEDULE_IN_RANGE")) {
std::string value = s;
if (value == "yes") {
allow_schedule_in_range = true;
} else if (value == "no") {
allow_schedule_in_range = false;
} else {
fprintf(stderr, "BENCH_ALLOW_SCHEDULE_IN_RANGE should be yes, no or not set");
return 1;
}
}
if (char* s = getenv("BENCH_PRINT_TRACE")) {
std::string value = s;
if (value == "yes") {
print_trace = true;
} else if (value == "no") {
print_trace = false;
} else {
fprintf(stderr, "BENCH_PRINT_TRACE should be yes, no or not set");
return 1;
}
}
if (char* s = getenv("BENCH_PAIR_COUNT")) {
char dummy;
if (sscanf(s, "%d%c", &pair_count, &dummy) != 1) {
fprintf(stderr, "BENCH_PAIR_COUNT should an integer");
return 1;
}
}
struct rusage start;
struct rusage end;
getrusage(RUSAGE_SELF, &start);
bench();
getrusage(RUSAGE_SELF, &end);
printf("%s,%d,%s,%lf,%ld\n", argv[0], pair_count,
(allow_schedule_in_range ? "yes" : "no"),
(end.ru_utime.tv_sec + end.ru_utime.tv_usec / 1000000.0) -
(start.ru_utime.tv_sec + start.ru_utime.tv_usec / 1000000.0),
total_rx_count);
return 0;
}
+536
View File
@@ -0,0 +1,536 @@
// -*- mode: c++; c-basic-offset: 4 indent-tabs-mode: nil -*- */
//
// Copyright 2016 Juho Snellman, released under a MIT license (see
// LICENSE).
//
// SPDX-License-Identifier: MIT
//
// A timer queue which allows events to be scheduled for execution
// at some later point. Reasons you might want to use this implementation
// instead of some other are:
//
// - A single-file C++11 implementation with no external dependencies.
// - Optimized for high occupancy rates, on the assumption that the
// utilization of the timer queue is proportional to the utilization
// of the system as a whole. When a tradeoff needs to be made
// between efficiency of one operation at a low occupancy rate and
// another operation at a high rate, we choose the latter.
// - Tries to minimize the cost of event rescheduling or cancelation,
// on the assumption that a large percentage of events will never
// be triggered. The implementation avoids unnecessary work when an
// event is rescheduled, and provides a way for the user specify a
// range of acceptable execution times instead of just an exact one.
// - Facility for limiting the number of events to execute on a
// single invocation, to allow fine grained interleaving of timer
// processing and application logic.
// - An interface that at least the author finds convenient.
//
// The exact implementation strategy is a hierarchical timer
// wheel. A timer wheel is effectively a ring buffer of linked lists
// of events, and a pointer to the ring buffer. As the time advances,
// the pointer moves forward, and any events in the ring buffer slots
// that the pointer passed will get executed.
//
// A hierarchical timer wheel layers multiple timer wheels running at
// different resolutions on top of each other. When an event is
// scheduled so far in the future than it does not fit the innermost
// (core) wheel, it instead gets scheduled on one of the outer
// wheels. On each rotation of the inner wheel, one slot's worth of
// events are promoted from the second wheel to the core. On each
// rotation of the second wheel, one slot's worth of events is
// promoted from the third wheel to the second, and so on.
//
// The basic usage is to create a single TimerWheel object and
// multiple TimerEvent or MemberTimerEvent objects. The events are
// scheduled for execution using TimerWheel::schedule() or
// TimerWheel::schedule_in_range(), or unscheduled using the event's
// cancel() method.
//
// Example usage:
//
// typedef std::function<void()> Callback;
// TimerWheel timers;
// int count = 0;
// TimerEvent<Callback> timer([&count] () { ++count; });
//
// timers.schedule(&timer, 5);
// timers.advance(4);
// assert(count == 0);
// timers.advance(1);
// assert(count == 1);
//
// timers.schedule(&timer, 5);
// timer.cancel();
// timers.advance(4);
// assert(count == 1);
//
// To tie events to specific member functions of an object instead of
// a callback function, use MemberTimerEvent instead of TimerEvent.
// For example:
//
// class Test {
// public:
// Test() : inc_timer_(this) {
// }
// void start(TimerWheel* timers) {
// timers->schedule(&inc_timer_, 10);
// }
// void on_inc() {
// count_++;
// }
// int count() { return count_; }
// private:
// MemberTimerEvent<Test, &Test::on_inc> inc_timer_;
// int count_ = 0;
// };
#ifndef RATAS_TIMER_WHEEL_H
#define RATAS_TIMER_WHEEL_H
#include <cassert>
#include <cstdlib>
#include <cstdint>
#include <cstdio>
#include <limits>
#include <memory>
typedef uint64_t Tick;
class TimerWheelSlot;
class TimerWheel;
// An abstract class representing an event that can be scheduled to
// happen at some later time.
class TimerEventInterface {
public:
TimerEventInterface() {
}
// TimerEvents are automatically canceled on destruction.
virtual ~TimerEventInterface() {
cancel();
}
// Unschedule this event. It's safe to cancel an event that is inactive.
inline void cancel();
// Return true iff the event is currently scheduled for execution.
bool active() const {
return slot_ != NULL;
}
// Return the absolute tick this event is scheduled to be executed on.
Tick scheduled_at() const { return scheduled_at_; }
private:
TimerEventInterface(const TimerEventInterface& other) = delete;
TimerEventInterface& operator=(const TimerEventInterface& other) = delete;
friend TimerWheelSlot;
friend TimerWheel;
// Implement in subclasses. Executes the event callback.
virtual void execute() = 0;
void set_scheduled_at(Tick ts) { scheduled_at_ = ts; }
// Move the event to another slot. (It's safe for either the current
// or new slot to be NULL).
inline void relink(TimerWheelSlot* slot);
Tick scheduled_at_;
// The slot this event is currently in (NULL if not currently scheduled).
TimerWheelSlot* slot_ = NULL;
// The events are linked together in the slot using an internal
// doubly-linked list; this iterator does double duty as the
// linked list node for this event.
TimerEventInterface* next_ = NULL;
TimerEventInterface* prev_ = NULL;
};
// An event that takes the callback (of type CBType) to execute as
// a constructor parameter.
template<typename CBType>
class TimerEvent : public TimerEventInterface {
public:
explicit TimerEvent<CBType>(const CBType& callback)
: callback_(callback) {
}
void execute() {
callback_();
}
private:
TimerEvent<CBType>(const TimerEvent<CBType>& other) = delete;
TimerEvent<CBType>& operator=(const TimerEvent<CBType>& other) = delete;
CBType callback_;
};
// An event that's specialized with a (static) member function of class T,
// and a dynamic instance of T. Event execution causes an invocation of the
// member function on the instance.
template<typename T, void(T::*MFun)() >
class MemberTimerEvent : public TimerEventInterface {
public:
MemberTimerEvent(T* obj) : obj_(obj) {
}
virtual void execute () {
(obj_->*MFun)();
}
private:
T* obj_;
};
// Purely an implementation detail.
class TimerWheelSlot {
public:
TimerWheelSlot() {
}
private:
// Return the first event queued in this slot.
const TimerEventInterface* events() const { return events_; }
// Deque the first event from the slot, and return it.
TimerEventInterface* pop_event() {
auto event = events_;
events_ = event->next_;
if (events_) {
events_->prev_ = NULL;
}
event->next_ = NULL;
event->slot_ = NULL;
return event;
}
TimerWheelSlot(const TimerWheelSlot& other) = delete;
TimerWheelSlot& operator=(const TimerWheelSlot& other) = delete;
friend TimerEventInterface;
friend TimerWheel;
// Doubly linked (inferior) list of events.
TimerEventInterface* events_ = NULL;
};
// A TimerWheel is the entity that TimerEvents can be scheduled on
// for execution (with schedule() or schedule_in_range()), and will
// eventually be executed once the time advances far enough with the
// advance() method.
class TimerWheel {
public:
TimerWheel(Tick now = 0) {
for (int i = 0; i < NUM_LEVELS; ++i) {
now_[i] = now >> (WIDTH_BITS * i);
}
ticks_pending_ = 0;
}
// Advance the TimerWheel by the specified number of ticks, and execute
// any events scheduled for execution at or before that time. The
// number of events executed can be restricted using the max_execute
// parameter. If that limit is reached, the function will return false,
// and the excess events will be processed on a subsequent call.
//
// - It is safe to cancel or schedule events from within event callbacks.
// - During the execution of the callback the observable event tick will
// be the tick it was scheduled to run on; not the tick the clock will
// be advanced to.
// - Events will happen in order; all events scheduled for tick X will
// be executed before any event scheduled for tick X+1.
//
// Delta should be non-0. The only exception is if the previous
// call to advance() returned false.
//
// advance() should not be called from an event callback.
inline bool advance(Tick delta,
size_t max_execute=std::numeric_limits<size_t>::max(),
int level = 0);
// Schedule the event to be executed delta ticks from the current time.
// The delta must be non-0.
inline void schedule(TimerEventInterface* event, Tick delta);
// Schedule the event to happen at some time between start and end
// ticks from the current time. The actual time will be determined
// by the TimerWheel to minimize rescheduling and promotion overhead.
// Both start and end must be non-0, and the end must be greater than
// the start.
inline void schedule_in_range(TimerEventInterface* event,
Tick start, Tick end);
// Return the current tick value. Note that if the time increases
// by multiple ticks during a single call to advance(), during the
// execution of the event callback now() will return the tick that
// the event was scheduled to run on.
Tick now() const { return now_[0]; }
// Return the number of ticks remaining until the next event will get
// executed. If the max parameter is passed, that will be the maximum
// tick value that gets returned. The max parameter's value will also
// be returned if no events have been scheduled.
//
// Will return 0 if the wheel still has unprocessed events from the
// previous call to advance().
inline Tick ticks_to_next_event(Tick max = std::numeric_limits<Tick>::max(),
int level = 0);
private:
TimerWheel(const TimerWheel& other) = delete;
TimerWheel& operator=(const TimerWheel& other) = delete;
// This handles the actual work of executing event callbacks and
// recursing to the outer wheels.
inline bool process_current_slot(Tick now, size_t max_execute, int level);
static const int WIDTH_BITS = 8;
static const int NUM_LEVELS = (64 + WIDTH_BITS - 1) / WIDTH_BITS;
static const int MAX_LEVEL = NUM_LEVELS - 1;
static const int NUM_SLOTS = 1 << WIDTH_BITS;
// A bitmask for looking at just the bits in the timestamp relevant to
// this wheel.
static const int MASK = (NUM_SLOTS - 1);
// The current timestamp for this wheel. This will be right-shifted
// such that each slot is separated by exactly one tick even on
// the outermost wheels.
Tick now_[NUM_LEVELS];
// We've done a partial tick advance. This is how many ticks remain
// unprocessed.
Tick ticks_pending_;
TimerWheelSlot slots_[NUM_LEVELS][NUM_SLOTS];
};
// Implementation
void TimerEventInterface::relink(TimerWheelSlot* new_slot) {
if (new_slot == slot_) {
return;
}
// Unlink from old location.
if (slot_) {
auto prev = prev_;
auto next = next_;
if (next) {
next->prev_ = prev;
}
if (prev) {
prev->next_ = next;
} else {
// Must be at head of slot. Move the next item to the head.
slot_->events_ = next;
}
}
// Insert in new slot.
{
if (new_slot) {
auto old = new_slot->events_;
next_ = old;
if (old) {
old->prev_ = this;
}
new_slot->events_ = this;
} else {
next_ = NULL;
}
prev_ = NULL;
}
slot_ = new_slot;
}
void TimerEventInterface::cancel() {
// It's ok to cancel a event that's not scheduled.
if (!slot_) {
return;
}
relink(NULL);
}
bool TimerWheel::advance(Tick delta, size_t max_events, int level) {
if (ticks_pending_) {
if (level == 0) {
// Continue collecting a backlog of ticks to process if
// we're called with non-zero deltas.
ticks_pending_ += delta;
}
// We only partially processed the last tick. Process the
// current slot, rather incrementing like advance() normally
// does.
Tick now = now_[level];
if (!process_current_slot(now, max_events, level)) {
// Outer layers are still not done, propagate that information
// back up.
return false;
}
if (level == 0) {
// The core wheel has been fully processed. We can now close
// down the partial tick and pretend that we've just been
// called with a delta containing both the new and original
// amounts.
delta = (ticks_pending_ - 1);
ticks_pending_ = 0;
} else {
return true;
}
} else {
// Zero deltas are only ok when in the middle of a partially
// processed tick.
assert(delta > 0);
}
while (delta--) {
Tick now = ++now_[level];
if (!process_current_slot(now, max_events, level)) {
ticks_pending_ = (delta + 1);
return false;
}
}
return true;
}
bool TimerWheel::process_current_slot(Tick now, size_t max_events, int level) {
size_t slot_index = now & MASK;
auto slot = &slots_[level][slot_index];
if (slot_index == 0 && level < MAX_LEVEL) {
if (!advance(1, max_events, level + 1)) {
return false;
}
}
while (slot->events()) {
auto event = slot->pop_event();
if (level > 0) {
assert((now_[0] & MASK) == 0);
if (now_[0] >= event->scheduled_at()) {
event->execute();
if (!--max_events) {
return false;
}
} else {
// There's a case to be made that promotion should
// also count as work done. And that would simplify
// this code since the max_events manipulation could
// move to the top of the loop. But it's an order of
// magnitude more expensive to execute a typical
// callback, and promotions will naturally clump while
// events triggering won't.
schedule(event,
event->scheduled_at() - now_[0]);
}
} else {
event->execute();
if (!--max_events) {
return false;
}
}
}
return true;
}
void TimerWheel::schedule(TimerEventInterface* event, Tick delta) {
assert(delta > 0);
event->set_scheduled_at(now_[0] + delta);
int level = 0;
while (delta >= NUM_SLOTS) {
delta = (delta + (now_[level] & MASK)) >> WIDTH_BITS;
++level;
}
size_t slot_index = (now_[level] + delta) & MASK;
auto slot = &slots_[level][slot_index];
event->relink(slot);
}
void TimerWheel::schedule_in_range(TimerEventInterface* event,
Tick start, Tick end) {
assert(end > start);
if (event->active()) {
auto current = event->scheduled_at() - now_[0];
// Event is already scheduled to happen in this range. Instead
// of always using the old slot, we could check compute the
// new slot and switch iff it's aligned better than the old one.
// But it seems hard to believe that could be worthwhile.
if (current >= start && current <= end) {
return;
}
}
// Zero as many bits (in WIDTH_BITS chunks) as possible
// from "end" while still keeping the output in the
// right range.
Tick mask = ~0;
while ((start & mask) != (end & mask)) {
mask = (mask << WIDTH_BITS);
}
Tick delta = end & (mask >> WIDTH_BITS);
schedule(event, delta);
}
Tick TimerWheel::ticks_to_next_event(Tick max, int level) {
if (ticks_pending_) {
return 0;
}
// The actual current time (not the bitshifted time)
Tick now = now_[0];
// Smallest tick (relative to now) we've found.
Tick min = max;
for (int i = 0; i < NUM_SLOTS; ++i) {
// Note: Unlike the uses of "now", slot index calculations really
// need to use now_.
auto slot_index = (now_[level] + 1 + i) & MASK;
// We've reached slot 0. In normal scheduling this would
// mean advancing the next wheel and promoting or executing
// those events. So we need to look in that slot too
// before proceeding with the rest of this wheel. But we
// can't just accept those results outright, we need to
// check the best result there against the next slot on
// this wheel.
if (slot_index == 0 && level < MAX_LEVEL) {
// Exception: If we're in the core wheel, and slot 0 is
// not empty, there's no point in looking in the outer wheel.
// It's guaranteed that the events actually in slot 0 will be
// executed no later than anything in the outer wheel.
if (level > 0 || !slots_[level][slot_index].events()) {
auto up_slot_index = (now_[level + 1] + 1) & MASK;
const auto& slot = slots_[level + 1][up_slot_index];
for (auto event = slot.events(); event != NULL;
event = event->next_) {
min = std::min(min, event->scheduled_at() - now);
}
}
}
bool found = false;
const auto& slot = slots_[level][slot_index];
for (auto event = slot.events(); event != NULL;
event = event->next_) {
min = std::min(min, event->scheduled_at() - now);
// In the core wheel all the events in a slot are guaranteed to
// run at the same time, so it's enough to just look at the first
// one.
if (level == 0) {
return min;
} else {
found = true;
}
}
if (found) {
return min;
}
}
// Nothing found on this wheel, try the next one (unless the wheel can't
// possibly contain an event scheduled earlier than "max").
if (level < MAX_LEVEL &&
(max >> (WIDTH_BITS * level + 1)) > 0) {
return ticks_to_next_event(max, level + 1);
}
return max;
}
#endif // RATAS_TIMER_WHEEL_H