And filter with Or filter inside

I'm trying to filter my indexed items by categories (Or) and then by filetype, so "And match this filetype, or this filetype" etc

My Data structure look like this:

{"FileType$$string": "xlsx","Id$$number": 15153,"FileYear$$string": "2020","FileCreation$$date": "2020-07-03T14:53:50Z","ShortDescription$$string": "Short description, lorem ipsum dolor sit amet, consectetur adipiscing elit Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit","ContentLink$$string": "15153_35198","PageUrl$$string": "https://foo.com/mypage/thispage","___types": ["Foo.Features.Common.Indexer.IndexedDownload","System.Object"
    ],"Title$$string": "Foo- Self-Service.xlsx","Category$$string": ["456"
    ],"$type": "Foo.Features.Common.Indexer.IndexedDownload, Foo.Features"
}

I have my current query set out at this:

            var query = SearchClient.Instance.Search<IndexedDownload>();
            if (filterQuery.Categories != null && filterQuery.Categories.Any())
            {
                foreach (var item in filterQuery.Categories)
                {
                    query = query.OrFilter(f => f.Category.In(new List<string> { item }, true));
                }
            }
            if (filterQuery.FileTypes != null && filterQuery.FileTypes.Any())
            {
                query = query.Filter(f => f.FileType.Match(filterQuery.FileTypes.FirstOrDefault()));
            }
            if (!string.IsNullOrEmpty(filterQuery.Years))
            {
                query = query.Filter(f => f.FileYear.Match(filterQuery.Years));
            }

If I try and change the FileTypes filter too:

                foreach (var item in filterQuery.FileTypes)
                {
                    query = query.OrFilter(f => f.FileType.MatchCaseInsensitive(item));
                }

It brings back file types it shouldn't, I am guessing because everything is wrapped up in an overall OR filter its bringing everything back based on those matches, running through Fiddler confirms this I think:

{"from":0,"size":9,"query":{"filtered":{"query":{"constant_score":{"filter":{"or":[
                     {"terms":{"Category$$string.lowercase":["453"
                           ]
                        }
                     },
                     {"terms":{"Category$$string.lowercase":["458"
                           ]
                        }
                     },
                     {"terms":{"Category$$string.lowercase":["456"
                           ]
                        }
                     },
                     {"terms":{"Category$$string.lowercase":["454"
                           ]
                        }
                     },
                     {"term":{"FileType$$string.lowercase":"xlsx"
                        }
                     },
                     {"term":{"FileType$$string.lowercase":"pdf"
                        }
                     }
                  ]
               }
            }
         },"filter":{"term":{"___types":"Civica.Features.Common.Indexer.IndexedDownload"
            }
         }
      }
   },"facets":{"FileType":{"terms":{"field":"FileType$$string"
         }
      },"FileYear":{"terms":{"field":"FileYear$$string"
         }
      },"Category":{"terms":{"field":"Category$$string"
         }
      }
   }
}

So what would be the way to get results based on being in one of the categories and in one of the file types too