Training: Drupal 7 Search Configuration Preview [April 16, 2014]
Want to learn more about Acquia’s products, services, and happenings in the Drupal Community? Visit our site: http://bit.ly/yLaHO5.
In this webinar, we’ll give you a sneak-preview of the step-by-step course “Drupal 7 Search Introduction”. In the online course, you can learn how to improve your site’s user experience for search and discovery on your site. With the right configuration you can increase search relevance and offer helpful search facets to allow your users to find the information they need quickly. In this webinar you will get:
• A quick overview for Drupal 7 Search configuration
• Sample lesson: A hands-on practical tutorial on improving your search results page
• A preview Acquia’s new Learning Management System and online courses available
• Webinar attendees will receive a discount code for the course.
Heather James: This is the Drupal 7 Search Configuration Preview Webinar. This is a preview of our search online course. I’m also going to walk you through some demos and some exercises so you’ll learn about what the course is like and you can actually learn a little bit about search in Drupal 7.
My name is Heather James and you can find me @LearningDrupal on Twitter. I’m the manager of Learning Services.
This webinar is for people who obviously want to know more about Apache Solr Search Integration in Drupal 7 and if you’re curious about Acquia’s Search Service and how it works and how you can get the most of out of it, and of course if you’re curious about Acquia’s training program.
Learning online is a new service that we’re offering and this gives us some self-paced options that we can offer to our various clients. It’s a little bit different from our classroom training. I still really love classroom training. We have a very busy calendar of events. So, please do check that out, but even still, for certain topics, it’s very hard to get a big class full of people together. Of course, something like a search or something like project management, and some of the other courses you’ll see coming online soon it’s difficult to get say 20 people or 10 people in a classroom for a topic like that. Whereas, we can get that online very easily and offer that to more people. Of course, you want to increase access to training so it’s not possible for our partners to make it to all corners of the globe. So, hopefully we can increase the reach that way for Drupal training as well. I’d still – like I said I really believe and love classroom-based training. If you have access to it it’s a lot of fun. We do more than just sit at the computers all day. There’s a lot of group interaction and ways that you sort of deal with all this new information in different ways. So, please – I’ve actually just written a blog so you can check that out if you’re more curious.
Our online training program is a little bit different. You can go to our training site and see some of our new Acquia product courses that are available. Then you can look at the catalogue and you can purchase the courses there. Most of our courses right now are just product specific.
We do have partners that I really love their services like Drupalize.Me. They have a video instruction which I love to use and work through but we find that some people also want alternatives. Our video instruction also includes the text details of what’s happening. We have some other content as well. There is, for example, in the search course, an interview with Nick Veenhof who is the maintainer of the Apache Solr Search Module and all of the activities are written out step by step. The videos are shorter than they would be. So, we don’t video everything, we actually just video say an introduction and then you follow through the steps and you can kind of mark off where you are.
We also have challenge exercises so that the material lets you sort of deal with things at your level. If you found something you’re able to get through more quickly you can give yourself a challenge exercise and get a little bit more out of it, but we try to streamline the course to show you what’s the most important thing you need to do, we know people’s time is very precious. We also have some fun quizzes that are in there, mainly just to sort of test your comprehension and give you a sense of progress, to have a feeling that and understanding of what you’ve actually learned.
Before we look at the course I wanted to take a second and think about search because I want to sort of bring to mind why you probably want to take this course. You’re probably already going through some – having some experience of having a site with a lot of content and people need access to that content or they want to get more out of it.
So, some research that KISSmetrics offer is a result of a survey that they did about people’s perception of their own behavior. People say they prefer to drill down. They like to see their options and keep their options open. You may notice this even yourself you might like to start - if you don’t know much about an area, you might like to start with a very broad topic and see the options available, and click to filter out those results. Even the filters start to give you some information about the content. So, some examples might come to mind if you think of shopping and you may know that you want to get a sweater but you’re not sure if you want it to be a cardigan or a pull over or you don’t know what color, and you don’t know the length. You like to see a variety of options.
Once you know what you want, it gets a little bit easier to make a very specific search but initially you might like to be very broad and the same probably goes for cars or for maybe a law document or something very specific that you’re looking for. So, what we can say is that users love facets. That’s something everybody starts to expect or has started to expect.
There’s another good article here on Intercom’s blog and it’s about Seven Things I Wish Every Search Box Did. So, I’d recommend reading that article. There’s a link you’ll be able to follow when you download the slides. This article is talking about some of the best search pages out there. It’s even funny to see something like auto-complete topped the list here, because Google only recently introduced that feature, maybe two years ago. Now, we’ve come to expect it and we don’t just want to see it on Google, we want to see it in all of our websites, as well as searching across multiple fields. It’s something that seems so obvious. Of course as a user you don’t really care if content is in the body field or maybe an attribute field of some content type, or even if it’s in an attachment. You just want access to that content. So, people just expect this.
People also expect that the results are ranked in some meaningful way and as well you can see some of these options here as well from number five people expect meaningful filters. We’re going to talk about filters in a little – next but what I wanted to say is that people expect that the filters relate to the content and the ranking relates to how they perceive the content.
So, in the example of ranking results it may be very important that more users are commenting on certain contents. You may want that to be more important. Maybe that’s what’s most important. Even if it’s something that’s two years old if there’s more user activity it may have more importance in your realm. Whereas, you may have very time sensitive information and you want some of the content that’s been published more recently to come up to the top. So, people just expect that to work and when it doesn’t and it’s unexpected it’s a very poor search experience.
As well, people expect typo correction. The last point says, “Mobile first,” right? People are typing away with their thumbs. They’ve got a baby in one hand or a drink in another, and a train ticket in another, and they just want the search to be a little forgiving.
The Nielson Group also offers some data about search dead ends. People want to make sure that – I mean, if they’re not sure what to look for they want to be offered alternatives and suggestions. They don’t want to be mocked like with silly pictures of like, “Oops, there’s nothing here.” That’s going to be a frustrating experience and the humor is not probably helping. So, the whole search experience should be very smooth even to the point where search just happens automatically. Maybe you’re looking at an article and you start to see more like this, “Oh, you’re interested in this content? You might want to see that.” So, the search experience is really built in throughout site and it’s something just almost automatic.
So, let’s look now at what Apache Solr can do. Now, this is really a preview of the modules that we’ll actually use in the course and what solutions they provide. Apache Solr has intelligent options as we spoke about. So, Apache Solr Auto Complete will provide suggestions and spelling corrections, and even tell you the number of the results per query. It’s something quite amazing. I probably will say this again as we do this presentation but the complexity here is masked because we will literally be able to install and enable a module and it just works out of the box. The Apache Solr has a lot of sister modules you can enable – or sorry, download and enable to extend what Apache Solr does because the index itself is very powerful.
Another option we’ll look at is files and attachments. Like I said end users don’t really care where related information is whether it’s in that attachment or a body field. Later on of course you can offer them filters when they have a better idea of what’s available but this just gives them access to all of your content. So many organizations have a lot of very important data and really great content that’s locked away in documents and PDFs. If they’re especially moving over from other systems into Drupal, something like this is going to reveal all that great content and value that people want.
Of course, sorting and filtering. So, like we said, users like to start broadly looking for some content and then drill down after seeing what options are available. So, the filters themselves will start to provide people information about the content you have. For example, these facets here might show exactly how much content is related to a specific country and what is the greatest – maybe the greatest population in this case is going to be city content, what’s the city with the greatest population on your site or the least population. These are available to us through using Apache Solr Sort and the Facet API.
The Facet API is sort of a hub and spoke model like some of our other sort of ecosystem modules in Drupal. It will actually let you choose how you want to display. So, we’ve got here a list with check boxes. You can have multi-faceted search or this slider. It depends on how you want to display the data and Apache Solr will be able to understand and make that more useful to you. As well, we want to provide those improved results and boosting.
In the course, we’d look at core search boosting which you can – there are options definitely with the basic core search. There’s far more options available with Apache Solr. You can even extend that with this Apache Solr Term Proximity. It literally looks at the distance between two or more terms. Is it much closer? Then we’ll rank that up in relevance. The fact is with site searches you don’t have the information that large search engines like Google have available to them. Google knows all about the inbound links and it’s got a huge index of the entire web and billions of pages. What you may have on your site is just a product description. You may have very small amounts of content. In that sense, tweaking those biases and making those – sorry, boosting those results can actually have a very important impact on your content. So, a lot of the things we’ll do just come with Apache Solr out of the box with result biasing and field biasing for example, but the Term Proximity is very useful.
As well people expect the results to be easy to read. They want to see the data as it should be displayed. If I just displayed location results as lat and long or latitude and longitude, it wouldn’t be very useful to me because I can’t really remember where’s 52 degrees [laughter]. I’m like, “Huh?” We do expect that data to show up. For example, on a map you want images if the images are in the content you want images to show up as images. So, that’s a really important part of the search experience that we will be looking at.
When we get to the custom code part of the course we can look at making intelligent suggestions. So, if someone does search for a location or maybe they even spell it wrong or maybe there isn’t any content directly there we can see and start to suggest locations that are near to their results and that’s done with a custom code example. All of this amazing technology as we’ll be using it is like I said, “The complexity is masked,” because we’ll be doing everything mainly within the UI with the exemption of this example. A lot of that is all based on the work that is in Apache Solr and I just wanted to go through really quickly and explain what an index is. This is really what makes Apache Solr very fast. Why wouldn’t we just pull from the database? Like we could make - with Views for example we can make a list of all the countries or each of the countries and use contextual filter, and display the results for each of those countries. Or we could use taxonomy terms and we can combine those things together, and we can do joins and bring it various data together. That actually would be very slow and it’s known to be a very slow approach.
Apache Solr actually creates indexes of that content. So, these inverted indexes – if we just take these three example documents over here on the right you can see that maybe these are just like body – let’s say this is like the body field or snippets from the body field of three different notes as an example. So, if you look at the words or where the words occur in each of those phrases like, “Where are you visiting? Are you visiting museums? I enjoy history museums.” You look at the index that results from that analysis or how it’s organized we see the word “where” shows up in the first document in the first position. The word “history” shows up in the third document in the third position. This is what really makes it so much more or much quicker to look up. This is how Apache Solr is doing some analysis and trying to understanding what’s there.
I guess it’s just useful to think about this so you understand that when we start to talk about optimizing those searches that you understand – you’re actually optimizing the index so, you have control over this. We can just look really quickly at how Apache Solr – how the index actually works. We’re using Acquia’s search service so we have some content on our site and we have to send that content to get indexed. As it gets queued and it comes back we can actually see what’s happening in the search index results here. This would happen even if you had Apache Solr installed locally on your machine you would still have to send this information over to your Apache Solr search server.
So, here we have for example every single time the cron runs which is basically a task. It’s basically a list of tasks that the server will run. It will actually, “Queued 200 items,” send them, and process them at once. So, we have over 2,000 items have to get processed so it’s going to take some time and a couple of returns to get that processed. So, you could see what’s happened here is that we’ve – 200 items have been indexed. We have 200 sent but haven’t yet been processed. At this time we can see 14% of that content has actually been sent to the server. So, we’re waiting for this round trip search and we’ve just run cron successfully about three minutes ago. So, what we’ll do here is refresh the index and see what has changed. So, right now we can see those items have come back 400 things have been indexed and we have a bunch remaining.
So, what we’re going to do is actually just choose to index all queued content. This will take some time and you can come back when it’s finish. It’ll take a few minutes. So, if your curiosity is piqued and you want to see how to use some of those features I’d like to show you a quick peek at the actual course. You can see what it’s like.
[Pause] So, this is what it looks like when you log in and you can see that there are instructions and there are various sections on the course. It basically, brings you through all the steps of setting up and installing your site, installing Solr, connecting your site to Acquia hosting. Basically, understanding sort of those - how things work and then you start to configure your search. Then you optimize your search, configure your facets, and then generally improve the user experience.
So, what we’re going to do now is actually look at some of those exercises and those activities. I thought it would be good to give you a bit of a preview of how things work. If I just show you the search features - I’m going to show you the site as it’s finished after we’re – after you’ve developed it within the course. I just want to show you basically what’s available and how things work.
[Pause] So, this is my sample site as it’s installed on Acquia hosting. You can also use the Dev Desktop and so you can work on it locally if you like. If you do it with Acquia hosting it’s fully online. You don’t have to install anything to do the course. So, we can conduct a simple search on the site. Actually before I show you that, let me just show you what comes with the site. You get some sample content because it’s just a little bit easier – we just want you to get started as quickly as possible. So, we have all this content in there and we’ve got some articles, and some cities, and we have for example, content types. We’ve got a custom content type in there. The city content type and this gives you sort of an idea of the type of data that’s available on the site. So, let’s conduct a quick search on the site and see what we get.
You can see that the results are laid out in a meaningful way as we spoke about before in which you learned how to use the Display Suite Module to actually arrange how the search results show up. You can see how the maps are appearing and we’ll look at using a couple of different modules. Let me actually show you what modules we’ll be using in fact.
So, I’m going to go back into the course here and show you the modules that are used. Of course, here are all of the Apache Solr search modules that you’ll be looking at and the display modules like Display Suite and we’ve also compared that to Rich Snippets which kind of makes like a Google looking type search just out of the box. Of course the Facet API and these excellent widgets that we saw like the slider or the check boxes, et cetera, and the location modules. So, we look at location and how we collect that data, and then how we display that data with Gmap.
So, let’s look at these search results here. We can see that they’re listed in order. Somehow they’re ranked. We don’t know yet. We’re going to look at how these are – the biases that are set but you can also see we have as well active still the core search results and you can just compare what those look like. Pretty straightforward with the core search results and you can compare what those advanced search options look like. You’ll notice when we did the search we set the Apache Solr Search as the default.
So, one thing we’re going to do right now is we’re going to make it – well, actually I just want to show you what it looks like with spell correction. If I search for “nurban” we’ll be getting a suggestion, “Did you mean, urban?” You can actually search for those results that way. We can also improve these results.
Now, what I’m going to do is I’m going to show you the Auto Complete Module. The Auto Complete Module will give people instant results. It will also make a suggestion if it maybe thinks it sees a typo and it will say, “Did you mean this?” It will show us – it will be that intelligent search experience like I said people just want now. The facility for this is kind of so easy. We’re just going to go to Modules Listing. I’m going to select autocomplete and enable it.
[Pause] I want to go back and I attempt to do a different search. You could see I was starting to get some suggestions, the word pops up, and we start to see how many times that phrase occurs in the various search results. So, this again completely hides all of the work that has gone into this but for you it’s literally making the choice that you want the feature, installing the module, enabling it and it just – it does what you want it to do.
So, let’s look at something that might be complex and difficult with other systems, Custom Search Pages. So, take for example you may want to specify a preset filter on all of your content. Set a scope for search. You may want to have a search for a specific country. That relates to our content but you may be able to bring to mind some examples from your own site where you know that people just want to search your book of content or they just want to search your product’s content. Or they want to see your articles, something like that - some way that you’re actually filtering on something.
You can even specify a taxonomy term and that’s what we’re going to do in this case. We’re actually going to make a search based on a particular field, taxonomy field. So, in order for us to do that we have to assemble the custom filter that we’re going to create and search for what filters – we’re going to look at what filters are available and then we’re going to show you how to actually assemble this. So, first thing we’re going to look at are the custom filter options available. One of them is the country field because that’s what is on our content type or the tags field that’s for example available in the article’s content type. We can look at the unique term ID as well. So, we’ll actually find out what is the unique term for a specific country. So, I’m going to do a quick demo making that search page, locating the field ID, and writing the custom filter, and creating it.
So, let’s look back on our search page again. We’re going to see that – right now we have a tab with the content tab and the site search tab. The content tab is of course the core search we’re keeping around. Now, we’re going to add our own search page. So, I’m going to call this my Country Specific Page, just for that example. The search environment is Acquia Search and we’ll call this the Ireland Search tab. The Custom Field is what we’ll actually be looking at and let’s fill out the tab. I’d like it to be something like search/Ireland and we’ll select a custom filter. This is sort of where you stop because you need to know what that custom filter is going to be. There’s no way you would know this off the top of your head. I certainly couldn’t remember it.
So, I’m going to go to my Administration List, Reports, and look at my Apache Solr search index. This will actually give you some insight into how Apache Solr is looking at your content and what’s available to actually filter on. The thing we’re looking for here is this country field. This right here, field_country, is actually the name of the field. I’m going to show you the content type just so you can see is actually the name of the field on the city content type that is my country taxonomy term field or term reference, I should say.
So, this is the filter we’re going to use. Now, what’s missing in this case is the ID. The way we find that out is by going back to look – I’m going to go back to structure and look at my taxonomy terms, and the list of terms that are available. There are quite a lot of countries here. It turns out that Ireland’s on the next page. We’ll get the term ID. You can click edit. You can see the term ID or up in your browser you’ll see. It’s 107 in this case. So, that will be the custom filter we’ll be working with. So, I’m going to save that. I would expect now if I click on this link that I’ll get my Ireland search.
So, what’s happening now is that I’ll start to get a limit of filters available. I’m going to search for houses for example in Ireland. Now, instead of getting all of the other countries you can see I’m only getting results within this one country. Although this is amusing it looks like this may be mismarked, maybe the latitude and longitude is wrong here. There you go. These are just some examples. Okay. That is basically how you make a custom search page. You probably have some ideas now that you’ve been watching it and how you might be able to use that in your own examples.
Next, let’s look at optimizing search. Improving bias is probably one of the most important things you’re going to be doing on your site. You know what you have to think about I suppose is how people are searching your content and what’s the most important thing about your content. For example, if you want more recent time sensitive content to show up to the top that’s a bias you’d want to improve. The only way you really can be able to do this is through actually starting to analyze your search logs once you get them so it’s not like build it, set it, and let it go. You probably want to build it, see how people are responding to it and constantly tweaking and improving your search, and taking in customer feedback, and watching their behaviors.
One thing I’ll say and we’ll talk about this when I do the demo is just to be very careful about how you tweak these boost settings. Don’t always put everything to the highest setting because then everything is just flat. What you want to do is really through testing and learning you’ll get an idea of how those results are improved. So, the factors are multiplying the relevance. So, 8.0 would be eight times boosted over the regular value. So, start very low and test as you go.
As we’re playing around you may want to just do something very exaggerated just so you can see your result but it’s not necessary on a live site. So, we’re going to first conduct a test search. We’re going to keep that page open just so you can see the direct change and we’re going to change the bias, and we’ll look at the index again, and see what changes are going to be made.
So, let’s start here under – yes, sorry. We’ll do a search actually that’s what I mean to say and we’ll search for [pause] flowers. We can see these two cities popping up. What we’re going to do is improve the search so that more recent content shows up. So, I’m going to keep this open and one thing I’m going to change is the date of this. [Pause] I’m going to edit that and change this to one year ahead. [Pause] Okay. At this time they’re exactly the same time I suppose so it won’t actually make any difference. We’ll just refresh there. You can see that Glovertown is up top and Ura is just underneath that.
So, let’s improve these search results. [Pause] Go back to configuration, search meta data, Apache Solr search, and under Acquia Search we can work on the bias. Here we’re going to set the result biasing. So, if something is more recently created we’re going to boost it by two. So, we’re going to double it and save it. So, we’re changing the settings and it also means we have to re-index because now one item hasn’t been sent to the server. So, we’ve actually changed the value of that item. So, we’ve changed the date of that item. So, it’s something new in the index and plus we’re changing the settings. So, we’re going to [pause] index the [QH] content which goes off to the server as you know. One thing you can do is actually go over to administration [pause], configuration, and cron, and you can run cron. So, that will take a few minutes and we’ll go back to configuration again. Search and look at the status of the index. So, one item has been sent and not processed. This is going to take a minute. So, when the cron job is complete and it has run around you can see finally zero items remaining and that means that updated content has been searched.
So, let’s go back and look at the results again. It was Glovertown versus Ura. I changed the date and changed the bias. So, we’ll do that same search again for flowers. You can see Ura which is more recent content is now coming up to the top. Actually all of the results now are rearranged based on how recent something was. So, that is just a little peek at bias.
There are so many options so there is the result biasing as we looked at. You can see how you can improve things again by how many comments, maybe more recent comments. Whether something is promoted to the front page and then you can even have specific field biases. So, there’s already some that are set by default. Some things - a header one or heading two in your body text. That’s going to get more boost and seen as more important. As well you can make certain types of content get improved as well. That’s just what Apache Solr is offering us out of the box which is quite brilliant.
So next, we’ll look at facets. Facets are the way that people can filter through their search. We can get a peek at the some of these facets, for example, on the site. [Pause] So, let’s look now at facets. Again, we’re talking about how people search and how people look for content on your site. Like I said, “You’ll have to listen to your users.” There’s a good article here from Smashing Magazine a couple of years ago still very effective and important is about looking at your query logs to define what those facets are going to be. Using what’s already going on in your industry, that’s obvious, right? You want to see what people expect and what’s done already. You want to choose something that’s very important to your users and important to you. So, how are people influenced to find certain content? Maybe you want to encourage certain content or behaviors and to get active in their community or to purchase something. It depends on what you’re working with. So, we can keep that in mind when we look at the city content type you can see there’s some information available. Location data or information about the population, et cetera.
Facet API offers you some options you can see here on the left. For example, you can have a list of links. You can have links with check boxes, and get that multi-faceted search. Or you can add additional modules like the Facet API Tag Cloud or the Facet API slider which will let you give people some options based on the kind of data they have. So, if it’s more important or more – if a term is showing up more in the Tag Cloud, it gets bigger. With Facet sliders it bases it on, for example, numerical information.
You can also do things to improve the user experience. As we’ll see, you can decide where you want those Facet blocks to appear. You can simply rearrange them in your layout. So, in our case we’ll put the Facet API slider up across the top so we have a nice long slider to use and you can put things in the sidebar for example. So, this is what we’ll be doing.
In the course you’ll actually set these in an exercise and then you’re given other challenge exercises to explore some of the other ones. This is again an example of we’re going to teach you something that is most expedient and quick, and efficient for you to learn. Then we’re going to give you a bit more if you have some more time. If you want to dig in a little bit deeper, we’ll give you some advice and some hints about where to go next.
I’m going to show you that really quick. Under Structure, Blocks. This is again with all the modules that need to be enabled and downloaded as we looked at in some of the course material. That’s assuming that you’ve simply installed and enabled the modules and they become available in this disabled list here.
So, I’m going to take the population slider and put that into the content list up here and make sure it’s at the top of the content list, and save it, and go back to our search page and refresh and see what changes we’ve made. Again, this is on the same search and you can see the sliders available, and I can limit to cities of a specific size or population like that. Again, this is something that just works out of the box. There are just so many different options and once you start getting familiar with it you start to master it.
The great thing is that because all this technical complexity is sort of pushed into the background you can focus on the stuff that’s important for your site, and the user experience, and how that search experience works. You don’t have to spend time writing custom code and doing complicated things. It basically is this easy.
So, what’s next? I’d recommend if you want to check out a webinar that my colleague did in August, you can watch to get some tips on how to configure Solr search for your site. Of course, you can take this course online. You can check out our Acquia Product Courses on our site and go through these step by step in much more detail. So, if you are new to Drupal and you’ve watched this webinar, you probably zoomed through a lot of things very quickly, you probably have a lot of questions. That’s basically what the course is all about. It’s taking you through these things. Self paced in a way that you can go at your own pace but hopefully in enough detail to get a lot out of it.
Thank you. I hope to see you online.