What we have learned going live with the Zimmo Prijswijzer

Note: before diving into this blog post, we suggest you try out the Zimmo Prijswijzer.

Last week (Tuesday 17/09/2019) Zimmo launched its Prijswijzer, an online tool to estimate the market value of a house or apartment in Belgium. The launch of this new estimation tool received quite some press coverage (Het Belang Van Limburg, De Standaard, Het Nieuwsblad), which resulted in a large amount of people using the tool during this first week (over 50,000 the first couple of days).

Rockestate has built the back-end of this application. Besides the predictive price model, this also consists of:

• The address auto-complete functionality;
• The 3D model and its visualization;
• The pre-filled building characteristics.

These steps are similar to the typical flow of an online onboarding process for products such as a home insurance or a mortgage loan.

In a previous blog post we explained the inner workings of the price estimation model. In this article, we try to uncover any insights we could draw from all these visits and what this tells about our data quality and our product’s functioning.

This is a clear case of how to optimize a conversion tunnel. We want a minimum of users to drop out during the process of entering the house details before getting an estimation. Put differently, we aim for a maximum of users to reach the final step of receiving the price estimate of their house.

An example of a conversion funnel.

Thanks to a large number of people visiting the Prijswijzer, we can calculate several conversion metrics which can serve as valuable indications of our data quality and the typical user behavior.

How many users find their address via the auto-complete form?

The first step in the process is the address auto-complete form. We have all Belgian addresses on file which are available in open data sources. If a user finds his address via this route, we don’t have to clean and match the address anymore (cf. blog post on address matching).

The main reasons why a user can’t find his address are:

• We don’t have the address on file, yet: for recent constructions, it often takes a while before the government adds the new address to its open dataset; our address list reflects this time lag.
• The address has multiple spellings (e.g. Sint-Pauluslaan vs. St. Pauluslaan, Avenue Louise vs. Av. Louise).

By analyzing the server logs, we observe that 4.2% of the users did not find their address via the auto-complete form. Out of this group, 64% ultimately got a matching address via the fallback input form (by clicking on “Het adres staat niet in de lijst”). Bottom line: 98.5% of users found the address of the house they wanted to estimate.

How many users confirm the 3D model of the house is accurate?

The second step in the process is for the user to confirm the 3D model of the building at the selected address. When a user confirms that the 3D model makes sense, we know that the open data of the address is most likely correct.

Out of all matched addresses, only 1.8% of the users don’t confirm the 3D model of the house. When someone rejects the 3D model, he or she gets a feedback form. Some of the comments we have received are listed below:

While we didn’t build a Natural Language Processing (NLP) model to analyze the hundreds of feedbacks we received in detail, we believe it is fair to state, on an empirical basis, that most of the time the user rejects the 3D model because it shows the neighbors’ house. This problem is most likely due to an incorrect address matching. It is not an issue of the 3D model quality.

What kind of houses are estimated?

We were wondering whether the houses estimated via the Prijswijzer were similar to those listed on Zimmo. We have therefore compared the distribution of house characteristics of Zimmo ad listings with a similar distribution of the Prijswijzer estimates.

Distributions of the house characteristics of all Belgian houses (blue) and the houses estimated by the users (orange).

The graphs above offer some interesting insights:

• only 35% of the houses listed on Zimmo have 0 touching sides (i.e. detached buildings such as villas) as opposed to 50% of the Prijswijer estimates;
• 47.8% of users knew the EPC-score(energy efficiency score) of the house they estimated. Out of this group, we notice that the estimated houses seem to have, on average, a lower EPC (the lower the value, the more energy-efficient) compared to the listings on Zimmo; -There doesn’t seem to be a significant difference in parcel area and living area between the Zimmo listings and the Prijswijzer estimates.

To measure is to know

The launch of the Zimmo Prijswijzer has been beneficial to Rockestate as it gave us an unbiased view of the raw conversion metrics of the application we have built. These numbers give us some invaluable insights which will directly contribute to the further improvement of our product. We already had a reasonably good idea of the fact that we are building a reliable solution. Thanks to the tens of thousands of users trying out the Prijswijzer during the first week after its launch, we now can put some concrete numbers on it.