'disable_delay' was static which is wrong as it is calculated using the per-device
bus speed. This patch turns 'disable_delay' into a per-device variable.

Reported-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Wolfram Sang <w.sang@pengutronix.de>
Acked-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: Darius Augulis <augulis.darius@gmail.com>
Signed-off-by: Ben Dooks <ben-linux@fluff.org>