>> How does equal cost multipath works (what algorithm is
>> used) when it load balances on default gateways? Is the routing
>> decision based by packet or by flow?

> This is how I understood it (and I would greatly appreciate any
> correction on this):
>
> . if you specify two routes with an equal metric value (using the
> "metric" or "preference" keyword), load balancing is done on a per
> connection basis;

No, it is done on a per route base (which is normally, but not always,
selected based on the IP of the destination). There is currently no
support to bind routes to individual streams. You can change it using
policy routing or fwmark routing though, but that is static.
 
>
> . if you specify the "equalize" keyword on two routes (which of course
> must have the usual triplet matching) then load balancing is done on a
> packet basis;

In this case the route is just recomputed for every packet. Without it
the route stays cached and bound to a specific nexthop (unless the nexthop
dies). Flow-balancing route means each new IP address that we
want to send packets to gets one of the two routes at random.
Use 'weight' to bias the randomness.

>
> . I tried to analyze the code (net/ipv4/fib_semantics.c) but I
> couldn't understand where the "weight" parameter came from, given also
> that the "weight" keyword in the ip user level command refers to
> multicast routes.

Weight it set by the nexthop parameter for ip route. 

===========

> I am about to add a second gateway routing entry to a RH7.0 box,
> that appears to have a 2.2.16 kernel with equal cost multi path and
> advanced routing options compiled into it. The box does not have rip,
> gated, or any other route checking program running on it.
> The two links to the Internet go out through different Ethernet cards,
> hubs, then gateway devices. The gateway devices as far as I am concerned are
> dumb devices that don't know if they are working or not.
>
> How will the 2.2 kernel's routing tables and the round robin affect
> of equal cost multi path handle the possibility of either the gateway
> device, or some other upstream link failing?

It won't, it needs an OSPF or similar daemon that downs/ups the interfaces
as needed.

In theory it could use the ARP neighbour reachability information for this,
but that isn't implemented. A real routing protocol is more general purpose
anyway, it is not limited to a single hop.

> So, in a nut shell, since a) the isp's are not going to be peering with
> me and b) the gateway devices themselves aren't supported to give me
> even their status, I need to implement some route checking scripts to
> add or remove the bad default gateway if it happens to go down? Is my
> line of thinking correct, or am I missing some information about how
> osfp/rip enabled programs work, or is there another tool available to do
> this?

You don't need to change the routes, but up/down the interfaces as needed.
Interface up/down is currently the only routing fail-over event supported.
You could probably do that with a single user space daemon that regularly
checks links with a second route by ARP or pinging.

> It seems that packets are be directed to the first route with the lowest
> metric. In the case of >1 route with the same metric, traffic is always
> sent to the first route. If this is the case, what it the point of even
> having ECMP?

Load balancing, and fail-over with routing daemons that support it (which
is currently none as far as I know, a few people were writing their own
primitive daemons to just check for nexthop alive)

>
> Perhaps I must use gated/zebra or some other routing daemon?

Yes, but probably only without automatic load balancing (because at least
zebra does not seem to support Linux multipoint routes yet)


> CONFIG_IP_MULTIPLE_TABLES=y
> CONFIG_IP_ROUTE_MULTIPATH=y
>
> I don't have iproute/iptools on this box, the routing and ip assignments
> were done with ifconfig and route:

Multipath routing can only be configured using iproute's nexthop option, it's
different from having multiple routes.

-Andi