Usability Testing Planing

Meeting Agenda

  1. New features review – a11yfirst checker, show block
  2. Known bugs – double scroll, anchor
  3. Next feature items – link, image
  4. Second round of usability testing goals –  See U of I box folder (Login required)
  5. Recruiting testers for usability testing (Jemma and Dena)
  6. A11yfirst Editor deployment process to Library CMS – meeting with Library IT/Training materials(ie. “getting started” video)/ collaboration with DRES accessibility Badging training system
  7. Go live target date to WordPress dev and stage sites – Nov 30, 2017
  8. Versioning strategy – current one is 0.5.0/ how to inform users that the editor was updated.
  9. Demo site maintenance and update –  “save” and “edit” functionality
  10. Review A11yfirst help file –

Block Format Change & Milestone Release


Conceptual Model for block format

  • Menubox with check box for block format is getting complicated.
  • Two main categories of conceptual model re “block”
    • simple block such as p, pre
    • complex block such as template, block quote
  • Implied conceptual model in ckeditor is that their block format menu is “simple block”, so it does not allow nesting.  In fact, block format is complex block
  • Bottom line is why don’t we simply use block quote menu button and use block format menu – advantage is using CKeditor way.

Block Format Implementation

block format menu – no gutter design next to menu items – used rich combo box instead of menu list

Next Milestone Release

Next milestone release will be on the week of September 25 and the project website demonstration page will be updated accordingly. This milestone will included complete heading plugin, link plugin, instyle plugin, blockformat and accessibility checker.


Nick Hoyt, JaEun Jemma Ku

Comparative analysis of treatment of “paragraph” / “norrmal” behavior


In the July 21 meeting, we decided to compare different editor interfaces’ behavior around  what happens when things with different levels of block formatting and inline formatting are tagged with Normal / Paragraph styles.

I compared Word Online, Word desktop, Google Docs, Gmail, Adobe Dreamweaver, the Confluence Wiki (used by, Drupal 7’s CKEditor implementation,’s TinyMCE implementation, and Box Note.

The analysis of the findings are below, followed by a table of data.

Strong standards:

  • Every editor surveyed files “normal” (or “body” or “paragraph”) with the headings.
  • Every editor surveyed changes “headline” to its “normal” style when that is selected.
  • Every editor surveyed doesn’t remove a link that’s in place when “normal” is applied over something with a link in it.
What this result suggests:
  • These are behavior standards that people have reason to expect everywhere they see editor interfaces. That provides a strong established-behavior-based reason to follow these standards.
  • These behavior standards don’t cause accessibility problems that we’re trying to actively change people’s behavior with.  So we don’t have correspondingly strong behavior-altering reasons to break these standards.
  • We’re not trying to specifically block people from being able to create paragraphs the way we’re trying to specifically block people from being able to use the phrase “read more” in a link. “Read more” is getting active programming countermeasures to combat people’s link-making behavior. But we don’t have an accessibility problem caused by people using paragraphs to try to prevent.
  • Providing interfaces that follow established expectations does provide an accessibility benefit for people with cognitive disabilities, in addition to a usability benefit for everyone.

Moderate standards:

  • The majority of editors don’t remove “bold” formatting when “normal” is applied over something that is bold. 
  • Word and Google Docs both do remove “bold” formatting.
There are two ways to look at this result:
  • CKEditor’s default behavior mirrors the numeric majority of editors, so staying with its default behavior will reinforce the numeric majority’s definition of standard behavior.


  • Word and Google Docs have a large amount of user mind share between them, and people likely spend more time in Word and Google Docs than the other editors (giving them a higher “time percentage” than “number percentage” of behavior-setting exposure time).  

No standards:

  • Different editors disagree in 4 different ways on how to handle it when a “bulleted” item is then assigned to “normal.”
  • With no clear standard set, we wouldn’t be breaking any  clear expectations with whatever decision we make. 
There are two ways to look at this result:
  • We can choose to leave CKeditor’s default behavior as the default here.


  • If we’re looking to establish A11yFirst as something that’s clearly different than unmodified CKEditor, following Word’s behavior rather than unmodified CKEditor would be a way to emphasize the difference.

The raw data

What it’s called
Bulleted list
Inline bold / link
Word Online
(found in same group as headings)
Becomes “normal” style
Becomes “normal” style
Bold is removed and becomes “normal”. Link is not removed.
Word Desktop (Windows)
(found in same group as headings)
Becomes “normal” style
Becomes “normal” style
Bold is removed and becomes “normal”. Link is not removed.
Google Docs
“Normal text”
(found in same menu as headings)
Becomes “normal” style
Font and size become “normal” but list is still bulleted
Bold is removed and becomes “normal.” Link is not removed.
“Normal” (found in same menu as headings)
Becomes “normal” style 
No change
No change
Adobe Dreamweaver
“Paragraph” (found in same menu as headings)
Becomes “normal” style
Paragraph tag is nested within LI tag
No change
Confluence Wiki
“Paragraph” (found in same menu as headings)
Becomes “normal” style
No change
No change
Drupal 7 CKEditor 
“Normal” (found in same menu as headings)
Becomes “normal” style
Paragraph tag is nested within LI tag
No change
Publish. illinois WordPress WYSIWYG
(TinyMCE Editor)
“Paragraph” (found in same menu as headings)
Becomes “normal” style
No change
No change
Box Note
“Body” (found in same menu as headings)
Becomes “normal” style
No change
No change

Lessons Learned – Usability Testing


  • Nick Hoyt
  • Dena Strong
  • JaEun Jemma Ku

Discussion Topic

  1. Review/analyze three usability testing sessions – result patterns
  2. UX lunch club idea/feedback regarding UI design
  3. Block format discussion regarding Library requirements


  • Change/improve vague ” remove format” to specific ones such as “remove heading” or “remove style”
  • Change in tool bar configuration in the second row as the order of
    •  Bold button, Italic button, Inline style menu, remove format button, divider, Special character button, language button (see the attached image below)
  • Gather info about “review format/normal” from other editors such as google doc, MS word, Box) -> comparative analysis by Dena Strong
  • Next “lessons learned meeting from the rest of usability testing” will be on the week of August 14, in which all the usability testings are done.

Discussion and Suggestions from Usability testing

  • Users are confused how to remove “block quotation” style ->redundancy of ” “remove format” function throughout User interface such as “remove format button”, “T” with underline and x mark
  • “inline style” can become “menu” button” not “richcombo” button in CK source function for the consistency
  • Feedback to CKsource -> Language menu button is icon only label, which appears to be kanji character, user thinks that the icon
    • will translate the existing language, not tagging it
    • will add special characters
  • Discussed adding “Normal” menu option under “heading” menu. “Normal” menu is the same function as “remove format” under heading menu
  • Adding font size options such as ” larger” or “smaller” rather than pixel size would help to prevent the users using heading level as a visual style. -> need to confirm its implementation with the rest of group members
  • Trying to find a way to give users ability to remove specific formats in consistet and intuitive way

Project Check-in Meeting


  • Nick Hoyt
  • Dena Strong
  • JaEun Jemma Ku
  • Jon Gunderson


  1. Review/Updates on plugins
  2. Updating public version of a11yfirst editor
    • Merging feature branches or not
  3. Usability testing results
    • Review results of current testing
    • Planned usability testing sessions
  4. Design changes based on user testing
  5. Budget: Conferences
  6. Final project report submission
  7. Team members’ summer vacation schedule


  • Keep feature branches separate from master branch
  • JaEun Ku will write up the final project report and share it with the group for the review (Report deadline is Aug 20, 2017)

Action Items

  1. Schedule the separate usability testing result review session by Nick, Dena and JaEun
  2. Schedule the discussion session with the library regarding block format feedback

Review of Plugin Updates

Project Updates

  • Completed: Inline Style
  • Some or most work has been done: Heading
  • Work in progress, still need some requirements decisions: Block Format
  • Not yet started, need requirements, designs: Link, Image, Table, A11y Checker

Discussion Topic

  • Logistics, details re. how to clone and checkout a tagged version, such as the one mentioned above, so that it could be placed on the Library server.
  • Where we stand with our plugins and what is needed for moving forward:

Action Items

  • A11yfirst plugin license info and where to add copyright issues/ language(ex: JS file)
  • Meeting agenda next week – A11yfirst server setup, requirement documentation/workflow for links, Image and Allycheker

Next Three Major Steps

  1. Integrate what we’ve learned from the Heading plugin into, and move forward with, the Block Format and Inline Style plugins.
  2. Information design of Heading Help dialog: simple text explanation of main a11y issues — brevity is important.
  3. Work on the design of our a11y checker: what are the programming APIs available, e.g. Show Blocks plugin and Balloon Panel; and the design of the individual checking functions, starting with Check Headings.

Package Distribution

Role Definition

  • Release/Distribution Manager – JaEun Ku
  • Developer – Jon Gunderson, Nick Hoyt, Haaris and JaEun for library-specific template plugin
  • Design – JaEun Ku and Nick Hoyt

Developer Documentation

  • How to set up CKEditor
  • How plugins are organized
  • How branches are created

Development Work Flow

  • Submodule for each plugin repositories
  • Three git repos for the project:
    1. ckeditor-a11yfirst-dist
    2. plugins-dev
    3. plugins-dist

Distribution VS Development

File Naming Convention

Inconsistencies with naming conventions.

Ex: a11yFirst_……, a11yfirst-heading, a11y_fi

We should be using a11yfirst (all lower case) and hyphens (-) as opposed to underscore (_)

Ex: a11yfirst-heading as opposed to a11yfirst_heading