WebHare community

Consilio cannot find certain elements

I have this overview page:

https://www.slootsmid.nl/occasions/

If I search for the header text (“Onze occasions”), Consilio can find the overview page:

https://www.slootsmid.nl/zoeken/?words=Onze+occasions

However, when searching for an other word on that page, for example “Twinclip”, Consilio cannot find it anymore:

https://www.slootsmid.nl/zoeken/?words=twinclip

Even though it’s both on the overview page as well as (linked to) a detail page:

https://www.slootsmid.nl/occasion/details/1948/stiga-twinclip-55-s-b-gazonmaaier

Any idea why Consilio doesn’t index this? It’s like Consilio just ignores the occasions grid.

For WebHare sites, Consilio indexes each publisher document separately, so a link to the detail page doesn’t trigger indexing that detail page directly.

Judging by the URL structure, the occasions page and the detail page are two different documents in the Publisher, so the occasions subpages (like category pages) do get indexed because they are linked from the overview page. The detail pages aren’t linked from the main detail page URL (which triggers a 404), so they aren’t indexed as Consilio cannot find them.

To fix this, you can generate a list of all subpages if Consilio is requesting the detail index page, for example by checking the user agent header:

IF (GetWebHeader("User-agent") LIKE "Consilio/*")

BTW, Consilio doesn’t index the text of links, as usually the link text tells more about the page that is being linked to than the page that the link appears on. In this case, if you search for “twinclip”, you expect the detail page as a result, not the overview page.

Thanks for the replies!

One question though:

Which page are you referring to with “detail index page”?

?

The list of all subpages, would that just be a simple (ul/li) list of links to let Consilio know where the occasion detail pages (last link) are so the contents can be indexed?

By the “detail index page” I mean the page that would be served when requesting https://www.slootsmid.nl/occasion/details/ (i.e. the ‘link’ of the file object in the Publisher, which is the URL Consilio initially requests when indexing the document).

Yes, that’s exactly right. Any enumeration of links to subpages would do.

1 Like

Bit late but thanks :slight_smile:

This is what I have written eventually:

    STRING ARRAY subpath_tok := Tokenize(this->subpath, "/");
    IF (Length(subpath_tok) = 1 AND (GetWebHeader("User-agent") LIKE "Consilio/*" OR GetWebVariable("consilio") != ""))
      RETURN PTR this->RunConsilioOverview();
  MACRO RunConsilioOverview()
  {
    RECORD occasiondata := this->webdesign->GetAllOccasionsData();
    STRING detailspagebaseurl := this->webdesign->GetOccasionDetailsBaseLink();

    RECORD ARRAY occasions :=
        SELECT *
             , link := this->webdesign->GetOccasionLink(dbid, title, detailspagebaseurl)
          FROM occasiondata.occasions;

    LoadWittyLibrary(Resolve("occasions.witty"),"HTML-NI")->RunComponent("consilio", CELL[occasions]);
  }

The GetWebVariable is purely for easy testing/debugging:
https://www.slootsmid.nl/occasion/details/?consilio=1

Works like a charm, thanks again!