How to write robust type wrappers around Eigen::Matrix

Question

I wanted to introduce type wrappers for Eigen's 2d vectors for both possible component orderings. My very first try was this

using RowCol = Eigen::Matrix<Eigen::Index, 2, 1>;
using ColRow = Eigen::Matrix<Eigen::Index, 2, 1>;

This has the problem that it silently convert from either type to the other. I'd like to get a compile error if that ever happens. My next try was therefore

using Matrix_t = Eigen::Matrix<Eigen::Index, 2, 1>;
struct ColRow;
struct RowCol : public Matrix_t 
{
    using Matrix_t::Matrix_t;
};

struct ColRow : public Matrix_t 
{
    using Matrix_t::Matrix_t;
};

This already disallows conversion from compound types like vector<RowCol> to vector<ColRow> but due to Eigen::Matrix's default ctor Matrix(const EigenBase<OtherDerived> &other), this still allows direct conversions from RowCol to ColRow. Therefore I tried to delete the according ctors and assignment operators which is the best solution I've come up with so far.

struct ColRow;
struct RowCol : public Matrix_t 
{
    using Matrix_t::Matrix_t;
    RowCol(const ColRow&) = delete;
    RowCol& operator=(const ColRow&) = delete;
    RowCol& operator=(ColRow&&) = delete;
};

struct ColRow : public Matrix_t 
{
    using Matrix_t::Matrix_t;
    ColRow(const RowCol&) = delete;
    ColRow& operator=(const RowCol&) = delete;
    ColRow& operator=(RowCol&&) = delete;
};

This works quite ok. The only downside is that I now have to be more verbose at some call-sites

RowCol rc;
rc = rc.cwiseMin(...);  // doesn't compile because operator= is ambiguous, could be RowCol& operator=(const ColRow&) or ColRow& operator=(const RowCol&)
rc = RowCol{rc.cwiseMin(...)};  // must be explicit

Is there a way to improve on that without losing any type-safety?