Skip to content

THE TIMES USA

The News that Matters

Menu
  • ABOUT
  • CONTACT
  • LIFESTYLE
  • NATIONAL NEWS
  • BUSINESS
  • INTERNATIONAL NEWS
  • TECHNOLOGY
  • PRICE OF BUSINESS SHOW AUDIOS
Menu

Digging for Digital Gold

Posted on June 15, 2022 by admin

By Elizabeth Thede, Special for The Times USA

 

The discovery of gold in 1848 in California spawned the Gold Rush of 1849. But finding gold today has less to do with wading through streams and more to do with effective text retrieval across terabytes of data. Here’s how to get started – no Western migration required.

The first step is to build an index across the data. An index is just an internal tool that allows the search engine to quickly sift through terabtyes, the digital equivalent of panning for gold. A single dtSearch index, for example, can hold up to a terabyte of text across one or more data repositories.

There are no limits on the number of search indexes that dtSearch can build and instantly search. In a concurrent-access environment like an Intranet or Internet site, search can run in a completely stateless manner, making it very easy to scale. Multiple concurrent search threads can operate with instant response time. Each user can then review search results with highlighted hits.

To start indexing, just point to the data you want to index and dtSearch does everything else. No need to even tell the software what types of data it is working with. The software will automatically recognize whether it is working with PDFs, emails, web-based formats, compressed data like ZIP or RAR, Microsoft Office files such as PowerPoint, Excel, Access, Word, OneNote, etc.

The final index contains a compilation of each unique word and number across the indexed data and its position in the data. For continually evolving data, dtSearch can update indexes automatically as often as you like using the Windows Task Scheduler. Updating an index simply adds new data, deletes old data and reindexes modified data. Index updates do not lock out searching, so instant search—even instant multithreaded concurrent searching—can continue unaffected.

Turning to search options, for the ‘49ers, the critical determination was binary: gold or not gold. Luckily, dtSearch has moved beyond a simple binary framework to provide over 25 different types of search options. You can search for words or phrases in any type of Boolean and/or/not configuration: gold nuggets and (Sacramento Valley or San Francisco) and not fool’s gold. Or you can add a proximity element, such as taking the entire previous search request and adding a requirement that gold nuggets appear within 34 words of silver bars, or say 17 words before silver bars.

Concept searching finds thesaurus or user-defined synonyms. Add on fuzzy searching adjustable from 0 to 10 to accommodate potential typographical errors whether relating to old document scans or current email misstypings. That way, even if San Francisco is mistyped San Franmisco, you can still pick that up with a low level of fuzziness. By default, dtSearch will search the full text and metadata of all items, or you can limit one or more elements of a search to specific metadata.

After a search, dtSearch will automatically rank search results for display with highlighted hits by so-called vector-spaced relevancy-ranking. With that type of default ranking, if you search for silver or gold, and silver appears millions of times in the indexed data but gold just a few times, gold mentions would rank more highly and items with the densest gold mentions would rank even higher. Or you can customize ranking “on the fly” at search time, like giving silver a negative weight of 7 and gold a positive weight of 8 regardless of the terms’ prevalence in indexed data. Or you can choose to adjust the weightings if one or both terms appears in certain metadata or near the top of a file, for example.

Beyond word and phrase queries, dtSearch also supports number and numeric range searches, letting you search for any number between 1948 and 2022. And dtSearch also supports date and date range searches, letting you search for any date between July 4, 1949 and April 17, 2023 even if the dates appear in different formats, like July 11, 1949 and 5/18/21. And dtSearch can even find credit cards in data, just to make sure that that a stray credit card number that paid for a prospecting pan isn’t still sitting out there.

But now suppose someone has found gold and is trying to actually hide that fact in text. Notably, a search engine like dtSearch can also search through many types of hidden data. Some examples:

  • Metadata may be “hidden” so it is really hard to spot looking at a file in its native application, requiring an extraordinary amount of clicking around to locate. But all metadata is right there in the binary format of each file, and hence “clear as day” to a search engine.

 

  • Embedding documents inside other documents will also not work to obscure text. dtSearch can seamlessly parse multilevel container files, like an email with a ZIP attachment including a OneNote file with a fully embedded PowerPoint.

 

  • Saving a file with a misleading file extension also won’t work as a means of hiding text. You can have an Access database saved with an Excel spreadsheet extension and dtSearch will still recognize the correct file format and search it appropriately.

 

  • Black text against a black background, white text against a white background or gold text against a gold background is just text for a search engine.

 

  • And dtSearch can even identify “image only” PDFs. These are files that look externally like normal PDFs, but really consist of just a straight-up picture, with no actual digital text. dtSearch can flag those at indexing time so you know you need to run them through an OCR application like Adobe Acrobat to make them full-text searchable.

About dtSearch. dtSearch has enterprise and developer products that run “on premises” or on cloud platforms to instantly search terabytes of “Office” files, PDFs, emails along with nested attachments, databases and online data. Because dtSearch can instantly search terabytes with over 25 different search features, many dtSearch customers are Fortune 100 companies and government agencies. But anyone with lots of data can download a fully-functional 30-day evaluation copy from dtSearch.com to find buried digital gold.

RELATED: Kevin Price of the Price of Business show discusses the topic with Thede on a recent interview.

You Might Also Like...

  • Digging into Data – Unburied Treasure

    By Elizabeth Thede, Special for The Times USA Previously, I addressed how a search index…

  • Digital 2020 Marketing

    SEO Expert Danny Star, CEO of Websites Depot Inc., a web agency in Los Angeles, has announced…

  • Beyond Boolean Search

    By Elizabeth Thede, Special for The Time USA   Many people have heard of Boolean…

  • Tippy's Digital Tipping Platform Receives Endorsement

    Tippy, the digital tipping platform for the salon and spa industry, has received an endorsement…

  • TTUSA and PoB Digital Network on the Best in Books for 2019

    The Times USA is a part of the Price of Business Digital Network. They celebrate…

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

VIDEO: This Week’s Best of our Network

https://www.thetimesusa.com/wp-content/uploads/2022/07/Dramatic-Changes-in-the-Economy-are-Leading-to-Signs-of-Hope.mp4

GDPR Compliance

USABR does not collect data on its visitors.  For more information visit: https://www.usabusinessradio.com/contact-us/

Contact

Contact articles@usabusinessradio.net for more information on articles on this site. BMuyco@usabusinessradio.net for all other information.

Recent Articles

  • 5 Ways To Improve Productivity In An Engineering Company
  • Live Your Best Life Now: Tips to Help You Get Started
  • The US Supreme Court Overturns Roe vs. Wade
  • The Qualities of an Excellent Real Estate Postcard Design
  • Biden’s War on Oil Companies, How Will That Help the Economy?

The Secret to Making New Rich Books Work

The Secret to Making New Rich Books Work

Also in TTUSA

  • 5 Keys to Successful Sales on Amazon
  • The Fastest Growing and Quickest Declining Airports
  • Best Slot Winnings | Slot machines: randomness, algorithms and chance of winning
  • Labor Expert Speculates that Bernie Sanders Could be Biden’s Labor Secretary Choice
  • How Sal Shakir ALL IN Entrepreneur Podcast Helps Hundreds of Entrepreneurs overcome Hurdles

RSS The Daily Blaze

  • Lovable Moments Celebrates Launch With 20% Off
  • Leading Authority on Political Crimes on January 6 Hearings
  • A Brief Guide on Anti-Money Laundering Transactions Monitoring
  • Remembering Emmett Till
  • A Forty Million Dollar Fraud and Priceless Investor and Entrepreneur Lessons

The Leader in Business News

The Leader in Business News

RSS USA Business Radio

  • Photo Booth in Houston, Texas Celebrates Launch
  • The State of Bitcoin Today
  • Great Websites for Business News
  • Dramatic Changes in the Economy are leading to Signs of Hope and More
  • The Airline Industry Continues To Struggle

RSS USA Daily Times

  • Mobile Photo Booth in Houston, Texas Welcomes New Customers
  • A Company that Literally Puts Its Money Where Its Mouth Is
  • How To Increase The Motivation Level Of Employees At Work
  • 5 Tips for How to Stop Soil Erosion
  • How Do Roofing Specialists Train for the Job?

RSS USA Daily Chronicles.

  • How to Hire an SEO Firm
  • Hutchinson’s Shocking Testimony States Trump Officials Were Involved in Witness Tampering
  • Bitcoin Forex Trading Platform Review: The Best Bitcoin Trading Platforms
  • What Is the Best Way to Find a Great Mortgage Lender?
  • What Is the Cost of a New Roof With Tear Off?

RSS Price of Business

  • The State of Bitcoin Today
  • Leading Authority on Political Crimes on January 6 Hearings
  • Lovable Moments Celebrates Launch of Business
  • Villa VIP – A Spellbinding Mykonos Mansion for the Most Exclusive Holidays
  • Working Outside In The Summer – Can Your Business Afford The Distraction

RSS US Daily Review

  • Mobile Photo Booth in Houston, Texas Celebrates Launch
  • U-Haul’s Five Fastest Growing States in 2022
  • The Pros and Cons of Hiding Instagram Like Count
  • Remembering One of the Last War Heroes of the Greatest Generation
  • What is Enterprise SEO?

PoB Digital Network

US Daily Review

USA Business Radio

USA Daily Chronicles

USA Daily Times

The Daily Blaze

The Times USA

Price of Business

© 2022 THE TIMES USA | Powered by Superbs Personal Blog theme