Twitter LinkedIN Xing facebook

Showcasing Reinforcement Learning at Otto Group’s DevCon 2019 in Hamburg

Otto Group brought together experts from software development and quality assurance at the DevCon 2019 in Hamburg. Artem Budishchev, Senior Data Scientist at collectAI, joined the conference to illustrate how reinforcement learning allows for smart automated decision making. On that note – if you are an expert in reinforcement as well, you should definitely check out our career opportunities.

Understanding Reinforcement Learning

Reinforcement learning is an area of machine learning. Machine learning algorithms enable digital software applications to acquire knowledge – they represent the essence of what is meant by artificial intelligence.

AI models based on reinforcement learning analyze data sets, make decisions within a given range and evaluate the outcome. If the outcome is positive (this has to be defined by data scientists), the model will receive a reward. Repeating this procedure over and over again, the model will identify patterns and strategies to maximize the rewards. Thus, a certain behavior is reinforced.

Artem explained the fundamental mechanics of reinforcement learning at DevCon 2019

Visualizing the capabilities of Reinforcement Learning

Computer games are a great way to showcase the capabilities of reinforcement learning. Watch the video below to see a reinforcement learning agent play the game breakout. Over the course of hundreds of games, the model identifies strategies to maximize rewards. Ultimately, it even minimizes its own efforts by pushing the ball behind the blocks.

Moreover, computer games help to grasp pitfalls in the development of reinforcement learning agents. When conceptualizing agents for specific tasks, rewards have to be defined carefully to avoid unintended results. The second video shows the performance of an agent playing a boat racing game. As the agent does not only receive rewards for finishing the race but also for collecting bonus gems, it exploits the reward mechanic by only collecting gems – but it will never cross the finish line.

Optimizing Receivables Management with Reinforcement Learning

collectAI applies reinforcement learning to consistently improve KPIs in receivables management. Our models are set up as Deep Q Networks (DQN). This approach to reinforcement learning is characterized by multiple analytical layers.

A goal for the AI could be to optimize the realization rate of a client. This goal takes multiple events into account. When sending out payment reminders, the model tracks if a customer opens the email, accesses the payment page or pays the due amount. Each event inherits a predefined value which represents the reward.

collectAI applies reinforcement learning to optimize customer communication

The model now adjusts customer communication in a way that leads to higher rewards. In practice, the model might find out that the payment rate will be higher when sending emails at 10 am instead of 10 pm. In this case, the AI will prefer to send out emails in the morning to maximize the outcome.

“Reinforcement learning has become the first choice for building autonomous agents. The approach is highly effective as it resembles the human, dopamine-based way of learning. Furthermore, it enables machines not only to identify patterns in data but also to make related decisions. Thus, reinforcement learning will support more and more automated decision processes in businesses across the world.”


Artem Budishchev, Senior Data Scientist at collectAI

Connecting Software Development and Quality at DevCon 2019

The DevCon 2019 combines the Quality Conference and Developer Conference. Hosted annually at Otto Group’s Campus in Hamburg to foster knowledge sharing. Main topics include machine learning, blockchain, cloud computing, and big data. Many members of the Otto Group as well as external companies took the chance to exchange thoughts at the event – among them EOS, Hanseatic Bank, BonPrix and Microsoft.