Cause
By using ui->scrollArea->setWidgetResizable(true); you allow the scroll area to resize the widget automatically:
If this property is set to true, the scroll area will automatically resize the widget in order to avoid scroll bars where they can be avoided, or to take advantage of extra space.
Furthermore, you calculate the new size of the QLabel in a wrong way, i.e. you use the size of its pixmap, which in turn remains constant.
Solution
In order to achieve the desired effect, I would suggest you to:
Explicitly set the widgetResizable property to false:
ui->scrollArea->setWidgetResizable(false);
Make the new size of the QLabel dependent on its old size, instead of the size of its pixmap:
m_imageLabel->resize(m_scaleFactor * m_imageLabel->size());
Example
Here is a minimal example I have prepared for you in order to demonstrate how the proposed solution could be implemented:
#include <QApplication>
#include <QWidget>
#include <QBoxLayout>
#include <QPushButton>
#include <QScrollArea>
#include <QLabel>
struct Viewer : public QWidget
{
explicit Viewer(QWidget *parent = nullptr) :
QWidget(parent)
{
auto *l = new QVBoxLayout(this);
auto *scrollArea = new QScrollArea(this);
auto *btnZoomIn = new QPushButton(tr("Zoom In"), this);
auto *btnZoomOut = new QPushButton(tr("Zoom Out"), this);
auto *label = new QLabel();
qreal scaleFactor = 1.25;
label->setPixmap(QPixmap("qt-creator-logo.png"));
label->setScaledContents(true);
scrollArea->setBackgroundRole(QPalette::Dark);
scrollArea->setWidget(label);
scrollArea->setWidgetResizable(false);
l->addWidget(scrollArea);
l->addWidget(btnZoomIn);
l->addWidget(btnZoomOut);
connect(btnZoomIn, &QPushButton::clicked, [label, scaleFactor](){
label->resize(scaleFactor * label->size());
});
connect(btnZoomOut, &QPushButton::clicked, [label, scaleFactor](){
label->resize((2 - scaleFactor) * label->size());
});
}
};
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
Viewer w;
w.show();
return a.exec();
}
Note: This code requires the test image qt-creator-logo.png to be located in the build folder at runtime.
Result
As written, this code produces the following result:


