Designing a "query builder" for advanced search

20 Aug 2008 - 10:15am
5 years ago
18 replies
5288 reads
L.A. King
2008

Anybody out there ever built a interface that helps users build an advanced
search query? I am currently working on a site that has over 5 million
articles, and search is the primary way users find this information. A lot
of these users are librarians and they know how to do complex Boolean
searches on various meta data fields. Currently, the search engine is
optimized for this type of user, which is completely ignoring the user who
does not know how to construct these complex queries. They have built a
"query builder" interface that allows non-librarian users to construct
complex Boolean searches on multiple meta data fields, but nobody uses it
because it is terrible.

It is an interesting situation because users actually do use the advanced
search on this site, which is unusual. I know we need to optimize for how
users use simple search as well, and this is also being addressed. But
creating a query builder that allowed non-librarians to do complex meta data
field searches would be really beneficial to this user base.

Any body designed something like this, or could point me to some examples of
such an interface?

Thanks.

--
Larry King
lerble at gmail.com
http://extrafancy.net

Comments

22 Aug 2008 - 8:07am
Matt Nish-Lapidus
2007

Hi Larry,

I just designed something exactly like this.. it's our first version
and it's not perfect, but average users seem to get it well enough and
it will only get better as we continue to revise...

Here's the URL for a live implementation: http://opl.bibliocommons.com/search

We approached it from the perspective of teaching people a little
about the boolean search syntax.. so instead of having a free form
search field and a separate form for advanced search we use the form
to construct a search string. Anybody that already knows the query
syntax can just type their own search into the text box.

Another example is the iTunes or Spotlight (on Mac) query builder.
You can make some pretty complex conditional queries using nothing but
a form.

Good Luck!

Matt.

On Wed, Aug 20, 2008 at 11:15 AM, Larry King <lerble at gmail.com> wrote:
> Anybody out there ever built a interface that helps users build an advanced
> search query? I am currently working on a site that has over 5 million
> articles, and search is the primary way users find this information. A lot
> of these users are librarians and they know how to do complex Boolean
> searches on various meta data fields. Currently, the search engine is
> optimized for this type of user, which is completely ignoring the user who
> does not know how to construct these complex queries. They have built a
....
>
> Any body designed something like this, or could point me to some examples of
> such an interface?
>
> Thanks.
>
> --
> Larry King

--
Matt Nish-Lapidus
work: matt at bibliocommons.com / www.bibliocommons.com
--
personal: mattnl at gmail.com
twitter: emenel

22 Aug 2008 - 9:15am
martinpolley
2007

Hi Matt,

I'm designing something very similar right now, and the design I cam up with
is very similar to yours :)

So I'm curious how you came to the solution whereby the user can enter their
own query or add to the one generated from the form (rather than making it
read-only, like Google advanced search).

I wrestled with the problem that the contents of the text box don't always
match what appears in the form. And with what should happen when the user
does something in the form after typing something in the text box. (We
decided that it was too complicated and that making it read-only was a
better bet. But that doing it this way enables the user to take what they
learned from this and use it in the regular, non-advanced search.)

I'm interested to hear your thoughts.

Cheers,

--
Martin Polley
Technical writer, interaction designer
+972 52 3864280
<http://capcloud.com/>

On Fri, Aug 22, 2008 at 4:07 PM, Matthew Nish-Lapidus <mattnl at gmail.com>wrote:

> Hi Larry,
>
> I just designed something exactly like this.. it's our first version
> and it's not perfect, but average users seem to get it well enough and
> it will only get better as we continue to revise...
>
> Here's the URL for a live implementation:
> http://opl.bibliocommons.com/search
>
> We approached it from the perspective of teaching people a little
> about the boolean search syntax.. so instead of having a free form
> search field and a separate form for advanced search we use the form
> to construct a search string. Anybody that already knows the query
> syntax can just type their own search into the text box.
>
> Another example is the iTunes or Spotlight (on Mac) query builder.
> You can make some pretty complex conditional queries using nothing but
> a form.
>
> Good Luck!
>
> Matt.
>
> On Wed, Aug 20, 2008 at 11:15 AM, Larry King <lerble at gmail.com> wrote:
> > Anybody out there ever built a interface that helps users build an
> advanced
> > search query? I am currently working on a site that has over 5 million
> > articles, and search is the primary way users find this information. A
> lot
> > of these users are librarians and they know how to do complex Boolean
> > searches on various meta data fields. Currently, the search engine is
> > optimized for this type of user, which is completely ignoring the user
> who
> > does not know how to construct these complex queries. They have built a
> ....
> >
> > Any body designed something like this, or could point me to some examples
> of
> > such an interface?
> >
> > Thanks.
> >
> > --
> > Larry King
>
> --
> Matt Nish-Lapidus
> work: matt at bibliocommons.com / www.bibliocommons.com
> --
> personal: mattnl at gmail.com
> twitter: emenel
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

22 Aug 2008 - 9:25am
Matt Nish-Lapidus
2007

Hi,

We came to this solution because we needed to accommodate existing
users that are already very comfortable with the query syntax, so a
free form search field was required... and since we were already doing
that, and the search syntax is more powerful than any form could be, i
decided that we should at least try to teach people about the query
syntax.

if a user doesn't want to learn it they can just use the form and hit
submit, but this way there is the opportunity for power users to
really get into searching.

we are also struggling with a way to update the form based on a typed
query, especially to allow people to edit a query after submitting it.
i have some ideas, but it's in early stages.

matt

On Fri, Aug 22, 2008 at 10:15 AM, Martin <martin.polley at gmail.com> wrote:
> Hi Matt,
>
> I'm designing something very similar right now, and the design I cam up with
> is very similar to yours :)
>
> So I'm curious how you came to the solution whereby the user can enter their
> own query or add to the one generated from the form (rather than making it
> read-only, like Google advanced search).
>
> I wrestled with the problem that the contents of the text box don't always
> match what appears in the form. And with what should happen when the user
> does something in the form after typing something in the text box. (We
> decided that it was too complicated and that making it read-only was a
> better bet. But that doing it this way enables the user to take what they
> learned from this and use it in the regular, non-advanced search.)
>
> I'm interested to hear your thoughts.
>
> Cheers,
>
> --
> Martin Polley
> Technical writer, interaction designer
> +972 52 3864280
> <http://capcloud.com/>
>
>
>
> On Fri, Aug 22, 2008 at 4:07 PM, Matthew Nish-Lapidus <mattnl at gmail.com>
> wrote:
>>
>> Hi Larry,
>>
>> I just designed something exactly like this.. it's our first version
>> and it's not perfect, but average users seem to get it well enough and
>> it will only get better as we continue to revise...
>>
>> Here's the URL for a live implementation:
>> http://opl.bibliocommons.com/search
>>
>> We approached it from the perspective of teaching people a little
>> about the boolean search syntax.. so instead of having a free form
>> search field and a separate form for advanced search we use the form
>> to construct a search string. Anybody that already knows the query
>> syntax can just type their own search into the text box.
>>
>> Another example is the iTunes or Spotlight (on Mac) query builder.
>> You can make some pretty complex conditional queries using nothing but
>> a form.
>>
>> Good Luck!
>>
>> Matt.
>>
>> On Wed, Aug 20, 2008 at 11:15 AM, Larry King <lerble at gmail.com> wrote:
>> > Anybody out there ever built a interface that helps users build an
>> > advanced
>> > search query? I am currently working on a site that has over 5 million
>> > articles, and search is the primary way users find this information. A
>> > lot
>> > of these users are librarians and they know how to do complex Boolean
>> > searches on various meta data fields. Currently, the search engine is
>> > optimized for this type of user, which is completely ignoring the user
>> > who
>> > does not know how to construct these complex queries. They have built a
>> ....
>> >
>> > Any body designed something like this, or could point me to some
>> > examples of
>> > such an interface?
>> >
>> > Thanks.
>> >
>> > --
>> > Larry King
>>
>> --
>> Matt Nish-Lapidus
>> work: matt at bibliocommons.com / www.bibliocommons.com
>> --
>> personal: mattnl at gmail.com
>> twitter: emenel
>> ________________________________________________________________
>> Welcome to the Interaction Design Association (IxDA)!
>> To post to this list ....... discuss at ixda.org
>> Unsubscribe ................ http://www.ixda.org/unsubscribe
>> List Guidelines ............ http://www.ixda.org/guidelines
>> List Help .................. http://www.ixda.org/help
>
>
>
>

--
Matt Nish-Lapidus
work: matt at bibliocommons.com / www.bibliocommons.com
--
personal: mattnl at gmail.com
twitter: emenel

22 Aug 2008 - 9:59am
Santiago Bustelo
2010

Some time ago we had a similar challenge for a backoffice application
targeting editors and developers. The following demo is fully
functional in Firefox and Safari (shows cosmetic bugs in IE):

http://www.icograma.com/demos/rulebuilder/

The editor allows building nested conditions via GUI or XML. As
expected, changes are reflected between modes. The XML is what the
backoffice app actually gets and understands.

Since we are allowing conditions to be added or removed, we can avoid
the dreaded long form with blank fields. The "Simple default new
rule" example shows the display of a simple default to get started.
This follows the same "design pattern" (looks fun to use & abuse
the term, I wanna play too) used in iTunes, several mail clients,
etc.

That is less confusing than showing a long form with several
"logically disabled" conditions (as "greater than... zero", or
"contains... [blank]"). Logically disabled conditions are a case of
programmer's design rather than good design. To a programmer, it
makes sense to think that there are zero pink elephants in the room.
The rest of mankind does not clutter their minds thinking about pink
elephants or, better said, their absence.
--
Santiago Bustelo // icograma
Buenos Aires, Argentina

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=32254

22 Aug 2008 - 11:52am
pnuschke
2007

My favorite query builder ever is used to segment people for marketing
campaigns:

http://www.sas.com/images/screenshots/solutions/mktauto/MA-5-1-screenshot.jpg

Probably my least favorite query builder is the one in Microsoft Access.

Perhaps relevant to your site, here's the advanced search for the Library of
Congress:

http://catalog.loc.gov/cgi-bin/Pwebrecon.cgi?DB=local&PAGE=Second

--
Paul Nuschke
Lead Design Researcher
ELECTRONIC INK(c)
www.electronicink.com

On Wed, Aug 20, 2008 at 11:15 AM, Larry King <lerble at gmail.com> wrote:

> Anybody out there ever built a interface that helps users build an advanced
> search query? I am currently working on a site that has over 5 million
> articles, and search is the primary way users find this information. A lot
> of these users are librarians and they know how to do complex Boolean
> searches on various meta data fields. Currently, the search engine is
> optimized for this type of user, which is completely ignoring the user who
> does not know how to construct these complex queries. They have built a
> "query builder" interface that allows non-librarian users to construct
> complex Boolean searches on multiple meta data fields, but nobody uses it
> because it is terrible.
>
> It is an interesting situation because users actually do use the advanced
> search on this site, which is unusual. I know we need to optimize for how
> users use simple search as well, and this is also being addressed. But
> creating a query builder that allowed non-librarians to do complex meta
> data
> field searches would be really beneficial to this user base.
>
> Any body designed something like this, or could point me to some examples
> of
> such an interface?
>
> Thanks.
>
> --
> Larry King
> lerble at gmail.com
> http://extrafancy.net
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

22 Aug 2008 - 11:19pm
Michael B. Moore
2008

This is a great topic.
I've done a couple of different systems for eDiscovery - systems that are
used by lawyers and paralegals to sift through the millions of documents and
emails that need to be coded for any big litigation project.

One thing I discovered when doing user research is that a Google type
approach wasn't going to work in this environment. In doing document
reviews, you have to be 100% certain you've a) looked at all the documents
and b) a set of documents is complete. So just showing the best matches
first didn't really excite anyone; some poor (but well paid) person has to
read through each one.

Like the original poster, I found that the majority of users were pretty
good at boolean searches, having had prior experience with Lexis/Nexis. But
they still wanted a fast way to find a specific document.

Here's a link to a screenshot of the search interface:
http://www.xerox-xls.com/ss-search.php

One thing I think worked well is that a text representation of the search
was constructed below the form, which users found helpful in understanding
where parentheses were going and what would be retrieved.

Another useful concept is to let users do a search within a search (use a
result set of one search as the starting point of a new search).

Lastly, for novice users, sometimes letting them do a series of simple
searches and then letting them combine or exclude results from the sets lets
them do fairly powerful searches without having to formulate one big complex
query. For example:

Cars costing < $30,000 = result set A
Car type is hybrid = result set B
Car brand is not Toyota = result set C
In set A + B + C = result set D

The National Library of Medicine used to have a public Medline search called
PubMed that used that approach. It was a bit slow, but pretty simple.

Like I said, it's a great topic.

Michael Moore
--
Michael B. Moore • Pure InfoDesign • 415.246.6690 M • www.pureinfodesign.com

22 Aug 2008 - 3:17pm
Carol Peterson
2008

I designed something similar to this years ago when working for a company called SilverPlatter. The company sold subscriptions to citation databases to libraries.

The software was designed to allow librarians to build search "wizards" that would encapsulate the librarians' advanced search knowledge. Library patrons would then use the wizards, supplying their search terms in prompt-driven forms, and the wizard would do the appropriate boolean search with the correct thesaurus terms.

We never ended up releasing the software, but it was well received during usabilty testing. The interface resembled a flowchart builder, and each "block" in the flowchart had search properties attached to it.

Although obviously SilverPlatter owns the design, I would be happy to discuss in general how we made various design decisions.

---------------------------------
Carol Peterson, Principal Usability Specialist
The MathWorks www.mathworks.com
carol.peterson at mathworks.com

22 Aug 2008 - 12:23pm
L.A. King
2008

Matt -

I like the idea of having the query builder create the query on the
fly inside the free form search field. That may be an approach we
take as it also helps to educate the user on how to do an advanced
search without using the query builder, and allow them to easily
refine the query in the free-form field. Although, I'd like the
search field to update and be visible the entire time, so when the
user makes a change, they can see the change happen in the free-form
field.

I agree with Paul, Microsoft Access: Worst Query Builder Ever. Much
easier just to write the SQL yourself, although Access makes it
really hard to get to that functionality.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=32254

22 Aug 2008 - 1:50pm
Shelley Price
2008

Hi,

One thing I think you might consider doing is putting a box
or call-out close to the query box that illustrates the what
each of the query operators means. We do this on our
Advanced Search, but also include a guided search with it.
That way, when they see a search constructed using the
templates, they have a visual cue to help them understand
the syntax that has just appeared in the box.

Unfortunately, all our products are subscription based, but
if anyone would like to see a screenshot, just let me know
and I'll send you a screen captures.

Cheers!
Shelley

"Matthew
Nish-Lapidu
s" To
<mattnl at gma Martin
il.com> <martin.polley at gmail.com>
Sent by: cc
discuss-bou discuss at ixda.org,
nces at lists. lerble at extrafancy.net
interaction Subject
designers.c Re: [IxDA Discuss]
om Designing a "query
builder" for advanced
search
08/22/2008
10:25 AM

Hi,

We came to this solution because we needed to accommodate
existing
users that are already very comfortable with the query
syntax, so a
free form search field was required... and since we were
already doing
that, and the search syntax is more powerful than any form
could be, i
decided that we should at least try to teach people about
the query
syntax.

if a user doesn't want to learn it they can just use the
form and hit
submit, but this way there is the opportunity for power
users to
really get into searching.

we are also struggling with a way to update the form based
on a typed
query, especially to allow people to edit a query after
submitting it.
i have some ideas, but it's in early stages.

matt

25 Aug 2008 - 10:06am
Santiago Bustelo
2010

@Paul Nuschke:
The screenshot of the SAS interface shows an algorithm builder, of
which queries are just one of the components.
Are there any screenshots showing how the "select" steps (i.e., the
queries) are built?
--
Santiago Bustelo // icograma
Buenos Aires, Argentina

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=32254

25 Aug 2008 - 10:12am
Santiago Bustelo
2010

I found Michael interface's text representation / query preview
really neat. Another useful preview would be the amount of results
the query will get (or even better, also the first few results).
Previewing the outcome of the query allows the user to work on the
expression only as necessary, and avoids getting no results after
building a complex query.

Alas, I am not thrilled by the use of "AND" and "OR" operators in
the GUI, as it can lead to confusion when having more than two
conditions or nested logic. Using "all/any of the folowing
conditions" benefits from a grammar that, by definition, avoids
those problems.

--

Santiago Bustelo // icograma
Buenos Aires, Argentina

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=32254

26 Aug 2008 - 12:26am
Michael B. Moore
2008

Agreed Santiago. In my experience with novice users, they almost always get
the meaning of Boolean operators exactly wrong. They tend to think of it
like they would speak it: "I want Red and Blue", meaning they want all the
Red and all the Blue, not things that are both Red and Blue.
On Mon, Aug 25, 2008 at 8:12 AM, Santiago Bustelo
<santiago at bustelo.com.ar>wrote:

>
> Alas, I am not thrilled by the use of "AND" and "OR" operators in
> the GUI, as it can lead to confusion when having more than two
> conditions or nested logic. Using "all/any of the folowing
> conditions" benefits from a grammar that, by definition, avoids
> those problems.
>
> --
>
> Santiago Bustelo // icograma
> Buenos Aires, Argentina
>
>
--
Michael B. Moore • Pure InfoDesign • 415.246.6690 M • www.pureinfodesign.com

26 Aug 2008 - 3:06am
martinpolley
2007

As always, this is a "know your users" thing. Most of the users of the query
builder I'm designing are highly technical types, who are familiar and
comfortable with AND and OR.

Cheers,

Martin Polley
Technical writer, interaction designer
+972 52 3864280
Twitter: martinpolley
<http://capcloud.com/>

On Tue, Aug 26, 2008 at 8:26 AM, Michael Moore <mmoore at pureinfodesign.com>wrote:

> Agreed Santiago. In my experience with novice users, they almost always get
> the meaning of Boolean operators exactly wrong....

>
> On Mon, Aug 25, 2008 at 8:12 AM, Santiago Bustelo
> <santiago at bustelo.com.ar>wrote:
> >
> > Alas, I am not thrilled by the use of "AND" and "OR" operators in
> > the GUI...
>

26 Aug 2008 - 8:26am
pnuschke
2007

I don't have more screenshots, unfortunately.

To give some context to that
screenshot.<%20http://www.sas.com/images/screenshots/solutions/mktauto/MA-5-1-screenshot.jpg>
..

Each block in that flowchart essentially represents one mini-query. These
blocks are essentially the WHERE clauses. The AND, OR, etc clauses are
usually built through the diagram. The end result, usually at the bottom of
the diagram, is the set of people to target with a specific campaign. You
can have multiple sets of people on one diagram, with different campaigns
for each.

On Mon, Aug 25, 2008 at 11:06 AM, Santiago Bustelo
<santiago at bustelo.com.ar>wrote:

> @Paul Nuschke:
> The screenshot of the SAS interface shows an algorithm builder, of
> which queries are just one of the components.
> Are there any screenshots showing how the "select" steps (i.e., the
> queries) are built?
> --
> Santiago Bustelo // icograma
> Buenos Aires, Argentina
>
>
> . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
> Posted from the new ixda.org
> http://www.ixda.org/discuss?post=32254
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help
>

26 Aug 2008 - 9:36am
Santiago Bustelo
2010

What I pointed out is not only about users being familiar with boolean
operators or not.

Boolean operators work in pairs. With more than two conditions,
grouping is needed. If parentheses are not placed, the grouping is
left to be done by the software when evaluating the expression, with
unpredictable results.

For instance, the following expression:
1 && 0 || 1 || 0 && 0

...may be evaluated as...
( 1 && 0 ) || ( 1 || 0 ) && 0 = 0
...or perhaps...
1 && (((0 || 1) || 0) && 0) = 0
...or maybe...
( ( (1 && 0) || 1 ) || 0) && 0 = 0

Actually, it evaluates it to 1 in FF3´s console. I guess it is grouped
as:
( 1 && 0 ) || 1 || ( 0 && 0 )

The ambiguity of the original expression is avoided (by definition)
with an unambiguous grammar . That is, one that doesn't allow mixing
AND with OR operators:

ANY of ( 1, 0, 1, 0, 0 ) are true = 1

ALL of ( 1, 0, 1, 0, 0 ) are true = 0

ANY OF (
ALL of ( 1, 0 ) ,
1,
ALL of ( 0, 0 )
) are true = 1

"ANY / ALL of ( ... )" is a form of prefix notation. Therefore it not
only reduces error and confusion, but is also far better suited for a
GUI that the written (infix) form.

--

Santiago Bustelo // icograma
Buenos Aires, Argentina

26 Aug 2008 - 11:05am
michel.milano
2005

santiago,
i haven't seen the term used yet, but what you are describing is called operator precedence.
it is more commonly encountered in math-with-numbers, where evaluating the following has only one correct result:

2 + 3 * 9 + 5

since boolean algebra isn't generally taught except to CS students, boolean operator precedence is not commonly known, but it exists just the same.

generally, the precedence is
1. parentheses
2. AND
3. OR

so the only "correct" interpretation is
( 1 && 0 ) || 1 || ( 0 && 0 )

26 Aug 2008 - 3:20pm
Santiago Bustelo
2010

> so the only "correct" interpretation is ( 1 && 0 ) || 1 || ( 0 && 0 )

An excellent use of the quotes, Michel. Precedence is not an axiom but
a mathematical convention, introduced to solve conflicts caused by
infix notation's grammar.

What I wanted to show is:
- infix notation has an ambiguous grammar,
- said grammar is not necessary nor well suited for a GUI,
- its interpretation can be unexpected by users (unless they have a CS
degree)
- conflicts caused by infix grammar can be avoided using prefix or
postfix grammar.

Using prefix or postfix grammar in the interface makes everyone's life
easier. User's don't need a CS degree to predict the outcome of an
expression. Developers don't need to work on precedence rules in order
to process the user's input. CS students will digress abut RPN ;-)

Regarding interaction design, this case is akin to the difference
between Verb-Subject grammar used in CLIs ( rm * ) and Subject-Verb
grammar used in GUIs ( select all, delete ).

--

Santiago Bustelo // icograma
Buenos Aires, Argentina

23 Aug 2008 - 4:59am
Dr. Clemens Lango
2008

hi larry,

my concept "visuos" might be of interest for your topic.
it contains a visual query paradigm that makes the formulation of boolean
queries *very* easy - also for non expert users.
the visuos site has just been completely relaunched and introduces this and
other concepts for knowledge work tasks.

it can be reached at http://visuos.com

cheers,
clemens lango.

----
dr. clemens lango
http://interactiondesign.de

Syndicate content Get the feed