Address space for a PCI device is assigned very early in boot (on PCs, it's typically done in the BIOS before the OS is even loaded). There are some devices that do allow the BAR size to be changed but the way to do it would be very specific to the device. Likely, you would need to do something physically to the device (set a jumper), or write to some non-volatile storage (NVRAM or something) on the device because that setting will need to survive a reset or power-cycle.
If your device actually supports changing the BAR size, the datasheet should explain how to configure it. As @prl said, for most devices, it can't be changed.
From the host side, there is a protocol used by the PCI root complex during device enumeration to discover how much space the device "wants" and then to tell the device what address range it has been assigned (and therefore should respond to). But as mentioned above, it's usually done very early in boot, because it requires coordination: it's easy enough for you to write to the device config registers to tell it it has more address space, but the memory controller and root complex need to know that too, or the actual memory references won't ever reach the device (and the system will likely malfunction too). And you can't just expand that address window willy-nilly because there's probably some other device that has already been assigned the memory just above and below your device.