Quite simple actually when there are multiple git repos in a project:
- Select the code tab.
- Click on the dropdown where it shows your repos.
- Select manage repositories.
- Click on the context dropdown next to the repo you want to delete.
- Select delete repository.


Edit (based on your edit):
TFS 2015 (update 1) indeed has an explicit minimum limit set for the number of Git repos contained within a TFS team project.
The TFS REST api contains functions which together can be used to delete a git repo but it does not delete the 'last' repo in the TFS project.
Here is the general API documentation
Two functions involved with deleting a git repo are:
Get a list of repositories
(GET VERB) https://{instance}/defaultcollection/[{project}]/_apis/git/repositories?api-version={version}
Which when given a project name returns json containing a list of git repos and their repo ID's.
Delete a repository which when given a repo ID will delete the specified repo.
(DELETE VERB) https://{instance}/defaultcollection/_apis/git/repositories/{repository}?api-version={version}
Unfortunately there is a catch when invoking the delete repo function:
{
"$id": "1"
"innerException": null
"message": "There must always be at least one repository per Team Project."
"typeName": "Microsoft.TeamFoundation.Git.Server.GitRepositoryMinimumPerProjectThresholdExceededException, Microsoft.TeamFoundation.Git.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
"typeKey": "GitRepositoryMinimumPerProjectThresholdExceededException"
"errorCode": 0
"eventId": 3000
}
Exception documentation on MSDN
Looking at SQL server on a lab instance I have (in my opinion unsupported if misused in a prod environment)
The TFS database contains a stored procedure which deletes git repos called prc_DeleteGitRepositories, it takes 4 arguments which I hunted down either in the database or hitting F12 on the web-ui.
It is invoked as follows:
EXEC prc_DeleteGitRepositories @partitionId=1,
@teamProjectUri ='vstfs:///Classification/TestProject1/cbcc3093-247d-448a-8c3b-f5d447fc8afa',
@repositoryId='4111286D-D066-4F3D-89B9-960055D678FE',
@deleterId='769254d3-1f13-431c-a580-1500dcbffbce'
That however throws the following in the scenario where only 1 git repo exists in the specified project:
Msg 50000, Level 16, State 1, Procedure prc_DeleteGitRepositories, Line 65
%error="1200013";%:<SERVERNAME>.TestCollection.dbo.prc_DeleteGitRepositories: There must always be at least one repository per Team Project.
So to summarize: a potential feature request best suited to uservoice as discussed in our comments on this Q/A.