Moving matrix transformations out of OF

I am working with the point cloud library which uses Eigen matrixes. I have prepared a lot of meshes that have been rotated and translated and all these changes are stored in a single ofMatrix4x4 per mesh.

My draw function for all of these meshes looks like this:

ofPushMatrix();
ofMultMatrix(modelLoaders[i].getDrawMatrix());
modelLoaders[i].drawVertices();
ofPopMatrix();

Where modelLoaders[i].getDrawMatrix()) retrieves an ofMatrix4x4. When I see all the meshes transformed by their matrixes they are drawn in the correct position and orientation, so far so good.

I want to use this matrix inside PCL, where it is used to transform the same meshes, PCL uses Eigen matrixes that are also row-vector (same as OF), but when I apply the matrixes to the meshes with PCL, the alignment is not correct.
I also tried to invert the matrix before I apply it with PCL, but this gives even stranger results.

Any idea on how I can take an OF matrix that is used for drawing and use it outside of OF while maintaining the orientation and translation?

Not fluent in PCL, but similar stuff has happend to me before, I am sure if you already saw good results with the model matrices on OF, it will not be the inverted matrix, probably just some row/column hack you need to pull of. https://stackoverflow.com/questions/17717600/confusion-between-c-and-opengl-matrix-order-row-major-vs-column-major#17718408

I have tried several variations, seeming the straight copy gets me closer, but it is still not correct, I am wondering if it is because OF uses an inverted y axis - would that make all my rotations around the y axis off, as well as the translations?

My function for getting the ofMatrix and applying it has been:

void transform(T cloud, ofMatrix4x4 matrix)
{
    Eigen::Matrix4f eigenMat;
    
    eigenMat << matrix._mat[0][0], matrix._mat[1][0], matrix._mat[2][0], matrix._mat[3][0],
    matrix._mat[0][1], matrix._mat[1][1], matrix._mat[2][1], matrix._mat[3][1],
    matrix._mat[0][2], matrix._mat[1][2], matrix._mat[2][2], matrix._mat[3][2],
    matrix._mat[0][3], matrix._mat[1][3], matrix._mat[2][3], matrix._mat[3][3];
    pcl::transformPointCloud(*cloud, *cloud, eigenMat);
}

And also

void transform(T cloud, ofMatrix4x4 matrix)

{
    Eigen::Matrix4f eigenMat;
	memcpy(&eigenMat, matrix.getPtr(), sizeof(float) * 16);
	pcl::transformPointCloud(*cloud, *cloud, eigenMat);
}

Neither give me the actual transformations I have created though. I am wondering whether I need to remove the projection matrix from my transformations.

Inverted y axis should not make a difference if your projection up vector is around (0,1,0 or 0,-1,0).

I am sure you already checked this out, the code is pretty well documented, can you run this? http://pointclouds.org/documentation/tutorials/matrix_transform.php

check out this important warning:

/* Reminder: how transformation matrices work :

       |-------> This column is the translation
| 1 0 0 x |  \
| 0 1 0 y |   }-> The identity 3x3 matrix (no rotation) on the left
| 0 0 1 z |  /
| 0 0 0 1 |    -> We do not use this line (and it has to stay 0,0,0,1)

Without knowing how PCL is rendering, or transforming point clouds (perhaps it needs a centered unit matrix transform to work, and then you apply the matrix and render elsewhere?). Did you try, in your transform function, to alter the row/column order?

would become

void transform(T cloud, ofMatrix4x4  **&** matrix)
{
    Eigen::Matrix4f eigenMat;
    
    eigenMat << matrix._mat[0][0], matrix._mat[0][1], matrix._mat[0][2], matrix._mat[0][3],
     etc
    pcl::transformPointCloud(*cloud, *cloud, eigenMat);
}

Also try to Log all values of the matrix before passing to the transform function, Log again once they are out, check how PCL loads models matrices and try to follow their examples. Hope this helps, I feel your pain when stuff like this doesnt work.

Thanks for the reply, I will give it all a go again, with more energy and clear head hopefully I can make it work. Good to know that the inverted y should not make a difference.