Commit 8c3d3622d9 introduced a new assertion,
but since it causes lp_test_conv failures remove it again and let's hope
we don't really hit bugs caused by the potentially bogus code (it is possible
the assert() caught some cases which work correctly too).