Tanner Thompson describes usage of a multi-armed bandit algorithm to drive conversions:
The functional idea behind a bandit algorithm is that you make an informed decision every time you assign a visitor to a test arm. Several bandit-type algorithms have been proved to be mathematically optimal; that is, they obtain the maximum future revenue given the data they have at any given point. Gittins indexing is perhaps the foremost of these algorithms. However, the trade-off of these methods is that they tend to be very computationally intensive.
This article doesn’t show any code, but it is useful for thinking about the problem.