Python Flask WTForms: Dynamic SelectField returning "Not a valid choice"

Question

First off, I know there are dozens of similar questions here. I have viewed most, if not all of them, and none of them have helped guide me to a solution. The following question is the most similar, but my implementation of "dynamic" is a bit different than theirs (more on this below): Flask / Python / WTForms validation and dynamically set SelectField choices

In short:

I have a form that is used to request a report from a network monitoring tool I built. The tool keeps track of all different kinds of statistics for various wireless networks. Here is the form class definition. My dynamic field is the ssidFilter selectField.

class RequestReportForm(FlaskForm):
    startDate = DateField('Start Date', validators=[DataRequired(), validate_startDate])
    startTime = TimeField('Start Time', format='%H:%M', validators=[DataRequired()])
    endDate = DateField('End Date', format='%Y-%m-%d', validators=[DataRequired(), validate_endDate])
    endTime = TimeField('End Time', format='%H:%M', validators=[DataRequired(), validate_allDates])
    ssidFilter = SelectField('SSID', default=('All', 'All'))
    reportType = SelectField('Report Type', validators = [DataRequired()], choices=[
                                                                ('rssi', 'RSSI vs. Time'), 
                                                                ('snr', 'SNR vs. Time'),
                                                                ('ClientCount', 'Client Count vs. Time'),

    ])
    selectLocation = SelectField('Locations', validators = [DataRequired()], choices=[ 
                                                                ('All','All'),
                                                                ('mainLobby', 'Main Lobby'), 
                                                                ('level1', 'Level 1'), 
                                                                ('level2', 'Level 2'),
                                                                ])
    submit = SubmitField('Generate Report')

I have already implemented Javascript to take the user-entered startDate and endDate fields, and run a query on my database by "fetch"ing another flask route in my app to return a list of all wireless networks (SSIDs) that were used within the date range they entered. Here is that route:

@app.route('/updateSSIDs/<startDate>/<endDate>', methods=['GET'])
def updateSSIDs(startDate, endDate):
    startDate = datetime.strptime(startDate, '%Y-%m-%d')
    endDate = datetime.strptime(endDate, '%Y-%m-%d')
    # Get a list of unique SSIDs that we have data for between the start and end dates selected on the form.
    SSIDs = getSSIDs(startDate, endDate)
    SSIDArray = []
    for ssid_tuple in SSIDs:
        ssidObj = {}
        ssidObj['id'] = ssid_tuple[0]
        ssidObj['ssid'] = ssid_tuple[0]
        SSIDArray.append(ssidObj)
    return jsonify({'SSIDs' : SSIDArray})

The variable SSIDArray looks like this before it is jsonify'd:

[{'id': 'Example Network 1', 'ssid': 'Example Network 1'}, {'id': 'Staff', 'ssid': 'Staff'}, ... ]

Here is how I am instantiating the form:

@app.route('/requestReport', methods=['GET', 'POST'])
def requestReport():
    form = RequestReportForm()
    form.ssidFilter.choices = getSSIDs(datetime.now(), datetime.now())

    if form.validate_on_submit():
        print("Valid form data:")
        print(form.data)
        flash(f'Received request for report from {form.startDate.data} at {form.startTime.data} through {form.endDate.data} at {form.endTime.data}', 'success')
        startDate = form.startDate.data
        startTime = form.startTime.data
        endDate = form.endDate.data
        endTime = form.endTime.data

        reportType = form.reportType.data
        locations = form.selectLocation.data
        ssid = form.ssidFilter.data

        # Put requested times into datetime objects
        startDateTime = datetime(startDate.year, startDate.month, startDate.day, startTime.hour, startTime.minute)
        endDateTime = datetime(endDate.year, endDate.month, endDate.day, endTime.hour, endTime.minute)

        # Generate report and redirect client to report.
        reportParameters = rpt.prepareReport_single(startDateTime, endDateTime, reportType, locations, ssid)        
        report = rpt.buildReport_singleLocation(reportParameters)       
        report = Markup(report)
        return render_template('viewReport.html', value=report)

Notice that I am populating my dynamic field, the form.ssidFilter.choices here by calling the same getSSIDs function that responds to my Javascript fetch call, but I'm passing in datetime.now() for both the start and end date. This is to initially show the user a list of wireless networks that are currently in use, but as soon as they change the dates, the list will update with a different set of networks.

And therein lies the problem: How can I set the list of acceptable choices (form.ssidFilter.choices) to contain the list of networks that comes back after a client enters the dates for the report?

Possible Solutions I'm exploring:

Reloading the page upon date selection to instantiate a new form with the dynamic data.
Keep a huge list of all the available choices at first, and then the choices will be dynamically filtered down via JS when the user changes the dates on the form.

Oh, and the form works fine if the SSID that is chosen happens to be an SSID that was in the list from the form.ssidFilter.choices = getSSIDs(datetime.now(), datetime.now()) statement. The issue only occurs when an item is selected that was not originally in the list of choices (which makes sense - I just don't know how to solve it).

Thank you for your time.

EDIT / Solution:

Thanks to @SuperShoot's answer, I was able to get this working. The key for me was to have the Flask route differentiate between the type of HTTP request - either GET or POST. Since I knew that the GET method was only used to retrieve the form and the POST method was only used to submit the filled out form, I could extract the startDate and endDate selections from the user, run the query to get the data, and update the choices field from my form class.

I had to do some additional validation as @SuperShoot also mentioned, but I did it a bit differently. Since my JavaScript code calls a separate route from my Flask app as soon as the end date is modified, the form had no responsibility to validate the date that was chosen. I implemented some validation in this other Flask route.

Here is my modified Flask requestReport route:

@app.route('/requestReport', methods=['GET', 'POST'])
def requestReport():
    form = RequestReportForm()
    form.ssidFilter.choices = getSSIDs(datetime.now(), datetime.now())

    if request.method == 'POST':
        startDate = datetime(form.startDate.data.year, form.startDate.data.month, form.startDate.data.day)
        endDate = datetime(form.endDate.data.year, form.endDate.data.month, form.endDate.data.day)
        # Update acceptable choices for the SSIDs on the form if the form is submitted.
        form.ssidFilter.choices = getSSIDs(startDate, endDate)

    if form.validate_on_submit():
        flash(f'Received request for report from {form.startDate.data} at {form.startTime.data} through {form.endDate.data} at {form.endTime.data}', 'success')
        startDate = form.startDate.data
        startTime = form.startTime.data
        endDate = form.endDate.data
        endTime = form.endTime.data

        reportType = form.reportType.data
        locations = form.selectLocation.data
        ssid = form.ssidFilter.data

        # Put requested times into datetime objects
        startDateTime = datetime(startDate.year, startDate.month, startDate.day, startTime.hour, startTime.minute)
        endDateTime = datetime(endDate.year, endDate.month, endDate.day, endTime.hour, endTime.minute)

        # Generate report and redirect client to report.
        reportParameters = rpt.prepareReport_single(startDateTime, endDateTime, reportType, locations, ssid)        
        report = rpt.buildReport_singleLocation(reportParameters)       
        report = Markup(report)
        return render_template('viewReport.html', value=report)

    else:
        return render_template('requestReport.html', title='Report Request', form=form)

And here is my updated updateSSIDs route which is called via Javascript when the form's end date is changed:

@app.route('/updateSSIDs/<startDate>/<endDate>', methods=['GET'])
def updateSSIDs(startDate, endDate):

    startDate = datetime.strptime(startDate, '%Y-%m-%d')
    endDate = datetime.strptime(endDate, '%Y-%m-%d')

    # Validate startDate and endDate
    emptyDataSet = {'SSIDs' : {'id ': 'All', 'ssid' : 'All'}}
    if startDate > endDate:
        return jsonify(emptyDataSet)

    if startDate >= datetime.now():
        return jsonify(emptyDataSet)

    if startDate.year not in range(2019, 2029) or endDate.year not in range(2019, 2029):
        return jsonify(emptyDataSet)

    # Get a list of unique SSIDs that we have data for between the start and end dates selected on the form.
    SSIDs = getSSIDs(startDate, endDate)
    SSIDArray = []
    for ssid_tuple in SSIDs:
        ssidObj = {}
        ssidObj['id'] = ssid_tuple[0]
        ssidObj['ssid'] = ssid_tuple[0]

        SSIDArray.append(ssidObj)
    return jsonify({'SSIDs' : SSIDArray})

This route is doing some basic checks to make sure the dates submitted are not entirely ridiculous before trying to retrieve the data from the database via getSSIDs, but I do some more thorough validation in the getSSIDs function.

SuperShoot SuperShoot · Accepted Answer · 2019-11-01T03:45:24

You can instantiate the form differently depending on whether the route handles a GET or POST request:

@app.route('/requestReport', methods=['GET', 'POST'])
def requestReport():
    form = RequestReportForm()
    if request.method == "GET":
        start = end = datetime.now()
    else:
        # validate start and end dates here first?
        start, end = form.startDate.data, form.endDate.data
    form.ssidFilter.choices = getSSIDs(start, end)
    ...

...although in the POST case, that uses the start and end dates before they have been validated. So one option is to first validate them inline inside the "POST" condition handling (where I've put the comment), or another is to override the .validate() method on the RequestReportForm.

This is the docstring of Form.validate():

        """
        Validates the form by calling `validate` on each field.
        :param extra_validators:
            If provided, is a dict mapping field names to a sequence of
            callables which will be passed as extra validators to the field's
            `validate` method.
        Returns `True` if no errors occur.
        """

A possible implementation is:

class RequestReportForm(FlaskForm):
    ...

    def validate(self, *args, **kwargs):
        """Ensure ssidFilter field choices match input startDate and endDate"""
        if not (self.startDate.validate(self) and self.endDate.validate(self)):
            return False
        self.ssidFilter.choices = getSSIDs(self.startDate.data, self.endDate.data)
        return super().validate(*args, **kwargs)

FlaskForm.validate_on_submit() first checks that the form is submitted, and then will call the custom .validate() method. The method first ensures that the start and end dates are valid and uses them to populate the expected possible values for ssidFilter before finally delegating validation back up the MRO.

I haven't run this code so let me know if any errors but hopefully I've got the idea across well enough for you to run with it if it suits.

Python Flask WTForms: Dynamic SelectField returning "Not a valid choice"

EDIT / Solution:

1 Answers