Skip to content
THE TIMES USA
Menu
  • ABOUT
  • CONTACT
  • LIFESTYLE
  • NATIONAL NEWS
  • BUSINESS
  • INTERNATIONAL NEWS
  • TECHNOLOGY
  • PRICE OF BUSINESS SHOW AUDIOS
Menu

Digging for Digital Gold

Posted on June 15, 2022 by admin

By Elizabeth Thede, Special for The Times USA

 

The discovery of gold in 1848 in California spawned the Gold Rush of 1849. But finding gold today has less to do with wading through streams and more to do with effective text retrieval across terabytes of data. Here’s how to get started – no Western migration required.

The first step is to build an index across the data. An index is just an internal tool that allows the search engine to quickly sift through terabtyes, the digital equivalent of panning for gold. A single dtSearch index, for example, can hold up to a terabyte of text across one or more data repositories.

There are no limits on the number of search indexes that dtSearch can build and instantly search. In a concurrent-access environment like an Intranet or Internet site, search can run in a completely stateless manner, making it very easy to scale. Multiple concurrent search threads can operate with instant response time. Each user can then review search results with highlighted hits.

To start indexing, just point to the data you want to index and dtSearch does everything else. No need to even tell the software what types of data it is working with. The software will automatically recognize whether it is working with PDFs, emails, web-based formats, compressed data like ZIP or RAR, Microsoft Office files such as PowerPoint, Excel, Access, Word, OneNote, etc.

The final index contains a compilation of each unique word and number across the indexed data and its position in the data. For continually evolving data, dtSearch can update indexes automatically as often as you like using the Windows Task Scheduler. Updating an index simply adds new data, deletes old data and reindexes modified data. Index updates do not lock out searching, so instant search—even instant multithreaded concurrent searching—can continue unaffected.

Turning to search options, for the ‘49ers, the critical determination was binary: gold or not gold. Luckily, dtSearch has moved beyond a simple binary framework to provide over 25 different types of search options. You can search for words or phrases in any type of Boolean and/or/not configuration: gold nuggets and (Sacramento Valley or San Francisco) and not fool’s gold. Or you can add a proximity element, such as taking the entire previous search request and adding a requirement that gold nuggets appear within 34 words of silver bars, or say 17 words before silver bars.

Concept searching finds thesaurus or user-defined synonyms. Add on fuzzy searching adjustable from 0 to 10 to accommodate potential typographical errors whether relating to old document scans or current email misstypings. That way, even if San Francisco is mistyped San Franmisco, you can still pick that up with a low level of fuzziness. By default, dtSearch will search the full text and metadata of all items, or you can limit one or more elements of a search to specific metadata.

After a search, dtSearch will automatically rank search results for display with highlighted hits by so-called vector-spaced relevancy-ranking. With that type of default ranking, if you search for silver or gold, and silver appears millions of times in the indexed data but gold just a few times, gold mentions would rank more highly and items with the densest gold mentions would rank even higher. Or you can customize ranking “on the fly” at search time, like giving silver a negative weight of 7 and gold a positive weight of 8 regardless of the terms’ prevalence in indexed data. Or you can choose to adjust the weightings if one or both terms appears in certain metadata or near the top of a file, for example.

Beyond word and phrase queries, dtSearch also supports number and numeric range searches, letting you search for any number between 1948 and 2022. And dtSearch also supports date and date range searches, letting you search for any date between July 4, 1949 and April 17, 2023 even if the dates appear in different formats, like July 11, 1949 and 5/18/21. And dtSearch can even find credit cards in data, just to make sure that that a stray credit card number that paid for a prospecting pan isn’t still sitting out there.

But now suppose someone has found gold and is trying to actually hide that fact in text. Notably, a search engine like dtSearch can also search through many types of hidden data. Some examples:

  • Metadata may be “hidden” so it is really hard to spot looking at a file in its native application, requiring an extraordinary amount of clicking around to locate. But all metadata is right there in the binary format of each file, and hence “clear as day” to a search engine.

 

  • Embedding documents inside other documents will also not work to obscure text. dtSearch can seamlessly parse multilevel container files, like an email with a ZIP attachment including a OneNote file with a fully embedded PowerPoint.

 

  • Saving a file with a misleading file extension also won’t work as a means of hiding text. You can have an Access database saved with an Excel spreadsheet extension and dtSearch will still recognize the correct file format and search it appropriately.

 

  • Black text against a black background, white text against a white background or gold text against a gold background is just text for a search engine.

 

  • And dtSearch can even identify “image only” PDFs. These are files that look externally like normal PDFs, but really consist of just a straight-up picture, with no actual digital text. dtSearch can flag those at indexing time so you know you need to run them through an OCR application like Adobe Acrobat to make them full-text searchable.

About dtSearch. dtSearch has enterprise and developer products that run “on premises” or on cloud platforms to instantly search terabytes of “Office” files, PDFs, emails along with nested attachments, databases and online data. Because dtSearch can instantly search terabytes with over 25 different search features, many dtSearch customers are Fortune 100 companies and government agencies. But anyone with lots of data can download a fully-functional 30-day evaluation copy from dtSearch.com to find buried digital gold.

RELATED: Kevin Price of the Price of Business show discusses the topic with Thede on a recent interview.

You Might Also Like...

  • TTUSA and PoB Digital Network on the Best in Books for 2019

    The Times USA is a part of the Price of Business Digital Network. They celebrate…

  • Tippy's Digital Tipping Platform Receives Endorsement

    Tippy, the digital tipping platform for the salon and spa industry, has received an endorsement…

  • Beyond Boolean Search

    By Elizabeth Thede, Special for The Time USA   Many people have heard of Boolean…

  • Digital 2020 Marketing

    SEO Expert Danny Star, CEO of Websites Depot Inc., a web agency in Los Angeles, has announced…

  • Digging into Data – Unburied Treasure

    By Elizabeth Thede, Special for The Times USA Previously, I addressed how a search index…

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

VIDEO: This Week’s Best of our Network

https://www.thetimesusa.com/wp-content/uploads/2023/09/The-United-States-Ten-Strongest-Allies_-A-Historical-Perspective.mp4

GDPR Compliance

USABR does not collect data on its visitors.  For more information visit: https://www.usabusinessradio.com/contact-us/

Contact

Contact articles@usabusinessradio.net for more information on articles on this site. BMuyco@usabusinessradio.net for all other information.

Recent Articles

  • Absolutely Insist on Enjoying Life
  • How I Learned to Stop Worrying and Love AI
  • The Importance of the Body Mass Index [BMI]
  • Beth Warford’s Commentary Feature on the Price of Business Digital Network
  • Cool eBikes With a So Cal Vibe

The Secret to Making New Rich Books Work

The Secret to Making New Rich Books Work

Also in TTUSA

  • Anti-depressants Raise New Health Care Concerns
  • Monopoly Slots Download | No deposit bonus for playing slot machines
  • Pointless Meetings Cost Hundreds of Billions of Dollars Annually
  • Sara Schulting Kranz Shares How To Thrive From Life’s Challenges and Traumas
  • This Media Whizz Is Rocking The Industry With New Amazing Ideas

RSS The Daily Blaze

  • Leading Tech Legal Authority on Biden Administration and Google
  • Fiduciary Company for Swiss Companies
  • Former White House Advisor Explains Why Economic Freedom Works
  • Changing Your Personality To Get Everything You Want In Life
  • Business take a Sudden Turn Against the “Woke Craze”

The Leader in Business News

The Leader in Business News

RSS USA Business Radio

  • How Small Businesses Can Compete With Large Ones
  • The Purification of Failure
  • Unmanned Surface Vehicles Become Oldest and Newest Fad in Warfare
  • Very Simple Solution to Houston’s Housing Problems
  • How To Deal With a Healthcare Insurers Denial; the Price of Doing Nothing

RSS USA Daily Times

  • Murder Suspect Accidentally Released from Jail in Indiana
  • How Will You Enjoy Forest Recreation This Year?
  • Why Chasing the Best Deal Is Very Expensive
  • Cynthia Appiah’s Commentary Feature on the Price of Business Digital Network
  • ARC Churches Shares Kelley Family Church Launch Story

RSS USA Daily Chronicles.

  • How to Earn Passive Income with PAMM Forex Brokers
  • Exclusive! Nutrition Coach on Breaking Free From Eating Disorders
  • The Importance of Monitoring Blood Sugar
  • Why Does a Well Trained Dog Live Longer?
  • Dare To Dream. Dare To Act. How You Can Create an Innovative Environment

RSS Price of Business

  • Murder Suspect Accidentally Released from Jail in Indiana
  • The Neuroscience of Hypnotherapy in Coaching
  • How Small Businesses Can Compete With Large Ones
  • Leading Tech Legal Authority on Biden Administration and Google
  • Embracing the Enlightened Workplace

RSS US Daily Review

  • Post Reporter on the Implications of UAW Strike
  • Embracing the Enlightened Workplace
  • 7 Unique Ways for Students To Get Some Extra Income
  • Retired General on Why North Korean Dictators Russia Visit Matters
  • Speaker McCarthy Declares a Formal Impeachment Inquiry on President Biden

PoB Digital Network

US Daily Review

USA Business Radio

USA Daily Chronicles

USA Daily Times

The Daily Blaze

The Times USA

Price of Business

© 2023 THE TIMES USA | Powered by Superbs Personal Blog theme