How to use openCV + kmeans + KMEANS_USE_INITIAL_LABELS in order to have predefined centres?

I have a grid of points filling a certain area (not neccessarly rectangular).
Now up to 5 points of interest get defined in this areas.
I am trying to use kmeans to now find the grid-points that are closest to each point-of-interest BUT in the end have the same/similar amount of grid-points assigned to each point-of-interest. (It might turn out that kmeans can not guaranty even quantity distribution)

Here is my code in which I use kmeans with OF.
The 3rd example function makeNewGridWithCenters that tries to solve the above problem but it is not working well. I am trying to seed the kmeans with cluster centres for which I need to use KMEANS_USE_INITIAL_LABELS.
But it seem I have to create a label-array the same size as my grid-point array. And fill the label-array with cluster indices (0-4 in my case).

Here an image of my grid layout. In order for function
void ofApp::makeNewGridWithCenters(vector<cv::Point2f> _myCenters)
to compile correctly I had to
Mat labels_mat = Mat(1, points.size(), CV_32S);
and I then just sequentially filled the label-array. Which does not make any sense.

So. How do I use kmeans with KMEANS_USE_INITIAL_LABELS correctly when wanting to predefine cluster centres?