'Unable to find control' when programmatically adding ASP.NET validation controls

I have been building a webpart that needs client side validation. I kept getting the error:

"Unable to find control id txtNotes referenced by the 'ControlToValidate' property"

Now most of the posts say just use the command to get the 'real' ID at runtime

this.rfvNotes.ControlToValidate = this.txtNotes.ClientID;

But this did not work, in the end the form I found worked was:

protected override void OnInit(EventArgs e)
    {
        base.OnInit(e);
        this.txtNotes.ID = "txtNotes";
        this.rfvNotes.ControlToValidate = this.txtNotes.ID;
        this.rfvNotes.Enabled = true;
        this.rfvNotes.Text = "* Required";
    }

The key step was to set the textbox ID manually, it seems that if this is not done it is the root of the error.