I get the error “Limit of total fields [???] in index [???] has been exceeded”

NOTE: This error was more common on versions prior to ElasticPress 5.0, as the plugin indexed all metafields. Since version 5.0, only metadata selected in the Search Fields & Weighting screen are indexed, unless you are using the ep_meta_mode filter with auto as value.

The error indicates that the total number of fields being stored by ElasticPress in ElasticPress.io exceeds the limit of what Elasticsearch is configured to accept. This does not mean that you have too many records in your Elasticsearch index. Instead, this error refers to distinct fields created by ElasticPress.

These fields are usually either metadata or taxonomies attached to Post objects in WordPress. Due to how ElasticPress indexes data for later search purposes, each field in WordPress is stored as 6 distinct fields in Elasticsearch. This means that, if your Elasticsearch is configured to allow up to 5,000 fields (the recommended limit), you can essentially only have 5,000 / 6 = 833 meta and/or taxonomy fields across all of your indexed content.

While that might sound like a lot, keep in mind that different custom post types can often create their own meta and taxonomy fields, and certain page builders as well as plugins (such as Advanced Custom Fields) create significant amounts of metadata which can quickly exceed the limits.

If you see an index fail silently or otherwise suspect based on your site’s content that this error might be a problem, you can confirm the error by running the following WP-CLI command to index while displaying all errors:

wp elasticpress sync --show-errors

The solution to this issue is to limit the fields of your content indexed by ElasticPress, which in turn will limit the number of distinct fields created in the Elasticsearch index. You can limit fields in two basic ways:

  • Remove all metadata and/or taxonomy fields from being indexed, and then add only the fields you want to index. This is a great approach if you want the absolute best performance and know exactly which fields you need to search or query on.
  • Alternatively, you can exclude certain fields from being indexed while leaving all others to be indexed. This is a great approach if you want flexibility down the road by indexing as much as possible, while removing content that will never need to be searched or queried on. If you use this approach, you can exclude meta using pattern matching, which can be quite useful to exclude repeater fields or other data which is rarely needed for search and can cause significant field bloat in the Elasticsearch index.

You can learn more about these approaches to limit which metadata is indexed to eliminate the field limit error in the article How to exclude metadata from indexing.