James Milligan, Lord Milligan