niXforums Forum Index
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   PreferencesPreferences   Log in to check your private messagesLog in to check your private messages   Log inLog in 
·  nixdoc.net ·  man pages ·  Linux HOWTOs ·  FreeBSD Tips ·  Forums
navigation Forum index » Databases » MySQL
err-report help me
Post new topic   Reply to topic Page 1 of 67 [996 Posts] View previous topic :: View next topic
Goto page:  1, 2, 3, ..., 65, 66, 67 Next
Author Message
Visolve DB Team
*nix forums beginner


Joined: 21 Jul 2006
Posts: 3

PostPosted: Fri Jul 21, 2006 5:11 am    Post subject: Re: FULL TEXT and Asian languages Reply with quote

Hi Peter,

Definitely using OR will slow up the performance of FULL TEXT searching.
Instead of using OR, you can try using UNION statement.

Hope this will be a fix for your issue.

Thanks,
ViSolve MySQL Support Team.


----- Original Message -----
From: "Peter Lauri" <lists@dwsasia.com>
To: "'JC'" <hiep@ee.ucr.edu>
Cc: <mysql@lists.mysql.com>
Sent: Thursday, July 20, 2006 4:21 PM
Subject: RE: FULL TEXT and Asian languages


Quote:
That is what I am doing right now, but it is not that fast if this system
would grow, and also it is not "ranking" the searches.

Right now I do something like this:

$searchwords = explode(" ", $searchstring);
foreach($searchwords AS $value) {
$Query.= "OR lajlaj LIKE '%$value%'"
}

If there are many search words, the OR will grow a bit, and OR are not
that
fast as I read somewhere.

/Peter

-----Original Message-----
From: JC [mailto:hiep@ee.ucr.edu]
Sent: Thursday, July 20, 2006 10:46 PM
To: Peter Lauri
Cc: mysql@lists.mysql.com
Subject: Re: FULL TEXT and Asian languages

don't know about indexing, but try to search: LIKE '%sentences%'

JC

On Thu, 20 Jul 2006, Peter Lauri wrote:

Best group member,

I have a problem. I was going to use FULL TEXT search for my Thai client.
It
is working smooth with English text and wordings, the indexing and search
works fine.

The problem with Thai text is that words are not separated with a white
space as in English and other languages. I think this screws up the
indexing, and complete sentences are classed as a word. Assume Thai
characters:

Thisisasentenceinthai. ButIcannotsearchforsentenceinthatsearch.

I want to search for "sentence", but can not. How can this be done? And
will
the indexing ever work?

Best regards,

Peter Lauri









--


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:
http://lists.mysql.com/mysql?unsub=mysql_support@visolve.com




--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.2/393 - Release Date: 7/19/2006




--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Neculai Macarie
*nix forums beginner


Joined: 13 Feb 2005
Posts: 8

PostPosted: Thu Jul 20, 2006 3:49 pm    Post subject: Re: FULL TEXT and Asian languages Reply with quote

Peter Lauri wrote:
Quote:
Best group member,

I have a problem. I was going to use FULL TEXT search for my Thai
client. It is working smooth with English text and wordings, the
indexing and search works fine.


"The FULLTEXT parser determines where words start and end by looking for
certain delimiter characters; for example, ' ' (space), ',' (comma), and '.'
(period). If words are not separated by delimiters (as in, for example,
Chinese), the FULLTEXT parser cannot determine where a word begins or ends.
To be able to add words or other indexed terms in such languages to a
FULLTEXT index, you must preprocess them so that they are separated by some
arbitrary delimiter such as '"'."

http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

--
<mack />


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Hiep Nguyen
*nix forums beginner


Joined: 11 Aug 2005
Posts: 11

PostPosted: Thu Jul 20, 2006 3:45 pm    Post subject: Re: FULL TEXT and Asian languages Reply with quote

don't know about indexing, but try to search: LIKE '%sentences%'

JC

On Thu, 20 Jul 2006, Peter Lauri wrote:

Quote:
Best group member,

I have a problem. I was going to use FULL TEXT search for my Thai client. It
is working smooth with English text and wordings, the indexing and search
works fine.

The problem with Thai text is that words are not separated with a white
space as in English and other languages. I think this screws up the
indexing, and complete sentences are classed as a word. Assume Thai
characters:

Thisisasentenceinthai. ButIcannotsearchforsentenceinthatsearch.

I want to search for "sentence", but can not. How can this be done? And will
the indexing ever work?

Best regards,

Peter Lauri









--


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Dominik Klein
*nix forums beginner


Joined: 15 Mar 2006
Posts: 35

PostPosted: Thu Jul 20, 2006 11:00 am    Post subject: Re: Problems with synchronisation master -> slave Reply with quote

Hi Thomas

Quote:
I try to configure replication from master to one slave for a database
with various table types (InnoDB, MyISAM) without stopping the master.

Therefore I make a mysqldump on the slave from master with this options:
mysqldump
-h master-db \
-udummyuser \
-pdummypass \
-v \
--all-databases \
--disable-keys \
--quick \
--single-transaction \

Single-transaction only affects innodb tables.

As single-transaction sets --skip-lock-tables automatically,
MyISAM-tables are not locked during the dump.
I think this is why you get the "row exists already" problem.

I'd suggest dumping innodb data and myisam data separatly. One mysqldump
WITH single-transaction (for innodb) and one without.
This leaves the problem of data being inserted in the meantime, as
master position will vary.

No idea at hand right now :(

Quote:
--master-data \
backup.file

an read the dump to the slave with:
mysql \
-u dummyuser \
-pdummypass \
backup.file

When I take a look to the backup.file, I see a line like this:
CHANGE MASTER TO MASTER_LOG_FILE='webdb1-bin.000170',
MASTER_LOG_POS=151635461;
caused by the option "--master-data".

But when I start the slave, there are soon the error message, that the
replication process will insert new row to a table, where this row exists
already.

Next try was, to delete all data in slave and then start replication from
the master from the beginning on (master-bin.000001), because we haven't
deleted any binlogs on master. After some time there appears an error
message, that there was an unsuccessful insert to an table, which doesn't
exist. Also the schema/catalog for this new table doesn't exist.

The question is now, why the creation of the new schema/catalog and the
table wasn't logged in the binlogs, so that they are not created via
replication before some inserts/updates are processed on them.

Maybe someone disabled log-bin temporarily when creating the db/table
(for tests or whatever) and forgot about this when inserting data later.

Quote:
Are the
binlogs not consistent? By the way, the "problem tables" are of type
MyISAM.

"Problem tables" = binlogged, but non-existent tables?
or
"Problem tables" = "row exists problem"-tables?

Regards
Dominik

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Peter Lauri
*nix forums beginner


Joined: 25 Nov 2005
Posts: 40

PostPosted: Thu Jul 20, 2006 10:53 am    Post subject: RE: FULL TEXT and Asian languages Reply with quote

Why can the Thai and Chinese not use regular sentences and word delimiter Smile
So I have to stick to my LIKE thing, just to erase the FULL TEXT index I
assume.

-----Original Message-----
From: Neculai Macarie [mailto:mack@eaccelera.ro]
Sent: Thursday, July 20, 2006 10:50 PM
To: mysql@lists.mysql.com
Subject: Re: FULL TEXT and Asian languages

Peter Lauri wrote:
Quote:
Best group member,

I have a problem. I was going to use FULL TEXT search for my Thai
client. It is working smooth with English text and wordings, the
indexing and search works fine.


"The FULLTEXT parser determines where words start and end by looking for
certain delimiter characters; for example, ' ' (space), ',' (comma), and '.'
(period). If words are not separated by delimiters (as in, for example,
Chinese), the FULLTEXT parser cannot determine where a word begins or ends.
To be able to add words or other indexed terms in such languages to a
FULLTEXT index, you must preprocess them so that they are separated by some
arbitrary delimiter such as '"'."

http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html

--
<mack />


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=lists@dwsasia.com


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Peter Lauri
*nix forums beginner


Joined: 25 Nov 2005
Posts: 40

PostPosted: Thu Jul 20, 2006 10:51 am    Post subject: RE: FULL TEXT and Asian languages Reply with quote

That is what I am doing right now, but it is not that fast if this system
would grow, and also it is not "ranking" the searches.

Right now I do something like this:

$searchwords = explode(" ", $searchstring);
foreach($searchwords AS $value) {
$Query.= "OR lajlaj LIKE '%$value%'"
}

If there are many search words, the OR will grow a bit, and OR are not that
fast as I read somewhere.

/Peter

-----Original Message-----
From: JC [mailto:hiep@ee.ucr.edu]
Sent: Thursday, July 20, 2006 10:46 PM
To: Peter Lauri
Cc: mysql@lists.mysql.com
Subject: Re: FULL TEXT and Asian languages

don't know about indexing, but try to search: LIKE '%sentences%'

JC

On Thu, 20 Jul 2006, Peter Lauri wrote:

Quote:
Best group member,

I have a problem. I was going to use FULL TEXT search for my Thai client.
It
is working smooth with English text and wordings, the indexing and search
works fine.

The problem with Thai text is that words are not separated with a white
space as in English and other languages. I think this screws up the
indexing, and complete sentences are classed as a word. Assume Thai
characters:

Thisisasentenceinthai. ButIcannotsearchforsentenceinthatsearch.

I want to search for "sentence", but can not. How can this be done? And
will
the indexing ever work?

Best regards,

Peter Lauri









--


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Dilipkumar
*nix forums beginner


Joined: 08 May 2006
Posts: 18

PostPosted: Tue Jul 18, 2006 7:52 am    Post subject: Re: load-file() doesn't work [SOLVED] Reply with quote

Hi,

If you have like insert statments try using this as

login into mysql
mysql -u root -p
use database-name

\. /tmp/filename.txt


Thanks & Regards
Dilipkumar
----- Original Message -----
From: "Marc Alff" <marc.alff@comcast.net>
To: <fleet@teachout.org>
Cc: <mysql@lists.mysql.com>
Sent: Monday, July 17, 2006 11:21 PM
Subject: Re: load-file() doesn't work [SOLVED]


Quote:

Hi Fleet


fleet@teachout.org wrote:
SHORT SYNOPSIS:
The statement
INSERT INTO table (blob_column) VALUES
(load_file('/home/somebody/image.jpg'));

Produces no error; but fails to load the image file.

SOLUTION:
The image file (or other binary file, I assume) MUST reside in / or /tmp,
ie /image.jpg or /tmp/image.jpg. (At least in MySQL 3.23.36)

I *hope* this is a bug!

- fleet -



Just to check, is the /home partition actually mounted on the server,
and at the same place ?

In other words, does /home/fleet/image.jpg exists :
- when seen from the host running mysql
- when seen from the host running mysqld

Marc.


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:
http://lists.mysql.com/mysql?unsub=dilipkumar_parikh@sifycorp.com


********** DISCLAIMER **********
Information contained and transmitted by this E-MAIL is proprietary to
Sify Limited and is intended for use only by the individual or entity to
which it is addressed, and may contain information that is privileged,
confidential or exempt from disclosure under applicable law. If this is a
forwarded message, the content of this E-MAIL may not have been sent with
the authority of the Company. If you are not the intended recipient, an
agent of the intended recipient or a person responsible for delivering the
information to the named recipient, you are notified that any use,
distribution, transmission, printing, copying or dissemination of this
information in any way or in any manner is strictly prohibited. If you have
received this communication in error, please delete this mail & notify us
immediately at admin@sifycorp.com

Watch India vs. England LIVE, Hot videos and more only on Sify Max! Click Here. www.sifymax.com

Get to see what's happening in your favourite City on Bangalore Live! www.bangalorelive.in


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
<fleet@teachout.org>
*nix forums beginner


Joined: 16 Jul 2006
Posts: 3

PostPosted: Mon Jul 17, 2006 5:23 pm    Post subject: Re: load-file() doesn't work [SOLVED] Reply with quote

On Mon, 17 Jul 2006, Marc Alff wrote:

Quote:
Just to check, is the /home partition actually mounted on the server,
and at the same place ?

In other words, does /home/fleet/image.jpg exists :
- when seen from the host running mysql
- when seen from the host running mysqld

In answer to the above: to the best of my knowledge. I haven't a clue how
to check that. I'm a user, not a tech. I have another small database
that produces an html output. It places it in my $HOME directory without
any specific path instructions; so I'd have to say mysql and mysqld *do*
see the directories.

mysql is running on my home PC. This particular PC is about 5 years old.
/home/fleet is my $HOME directory. I'm launching mysql from $HOME using:
[~]$ mysql -u root
mysql was installed (by someone who presumably knew what they were doing)
when I purchased the computer. I've never used it before now.

- fleet -



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Marc Alff
*nix forums beginner


Joined: 12 Mar 2006
Posts: 3

PostPosted: Mon Jul 17, 2006 5:03 pm    Post subject: Re: load-file() doesn't work [SOLVED] Reply with quote

Hi Fleet


fleet@teachout.org wrote:
Quote:
SHORT SYNOPSIS:
The statement
INSERT INTO table (blob_column) VALUES
(load_file('/home/somebody/image.jpg'));

Produces no error; but fails to load the image file.

SOLUTION:
The image file (or other binary file, I assume) MUST reside in / or /tmp,
ie /image.jpg or /tmp/image.jpg. (At least in MySQL 3.23.36)

I *hope* this is a bug!

- fleet -



Just to check, is the /home partition actually mounted on the server,
and at the same place ?

In other words, does /home/fleet/image.jpg exists :
- when seen from the host running mysql
- when seen from the host running mysqld

Marc.


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
<fleet@teachout.org>
*nix forums beginner


Joined: 16 Jul 2006
Posts: 3

PostPosted: Mon Jul 17, 2006 3:28 pm    Post subject: Re: load-file() doesn't work [SOLVED] Reply with quote

SHORT SYNOPSIS:
The statement
INSERT INTO table (blob_column) VALUES
(load_file('/home/somebody/image.jpg'));

Produces no error; but fails to load the image file.

SOLUTION:
The image file (or other binary file, I assume) MUST reside in / or /tmp,
ie /image.jpg or /tmp/image.jpg. (At least in MySQL 3.23.36)

I *hope* this is a bug!

- fleet -



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
sheeri kritzer
*nix forums Guru Wannabe


Joined: 19 Oct 2005
Posts: 243

PostPosted: Thu Jul 13, 2006 6:24 pm    Post subject: Re: create view and insert into problems Reply with quote

create view v_authornames as
select authorid, CONCAT(lastname,',',firstname)
from t_authors;

Or replace the middle term in the CONCAT function to whatever you want
to separate it -- ie, ' ' for a space, or just
CONCAT(lastname,firstname) to get output "KritzerSheeri".

-Sheeri

On 6/25/06, Andreas Bauer <andreas_bauer@arcor.de> wrote:
Quote:
Hello NG,

I have two tables in my mysql database
created with phpmyadmin:

t_authors:
1 authorid (primary key, auto_increment)
2 lastname
3 firstname

And a table named t_books, fields inside:

t_books:
1 bookid (primary key, auto_increment)
2 authorid (Typ:index, reference to authorid from t_authors done
with phpmyadmin)
3 title
4 subtitle

Now I want to create a view from t_authors, so that the fields
of lastname and firstname are one field with
the value inside: "lastname, firstname":

create view v_authornames as
select authorid, lastname || ', ' || firstname
from t_authors;

But this view created only an empty field named
lastname ||', ' firstname. How can I join this two
fields so that I get one and this value?

There is another problem of me inserting values sequently in
the two tables:


insert into t_authors (lastname, firstname)
values ('Meyers', 'Scott');

insert into t_books (authorid, title, subtitle)
values ('1'), 'Effektiv C++ Programmieren',
'50 Wege zur Verbesserung Ihrer Programme und Entwuerfe');

insert into t_books (authorid, title, subtitle)
values ('1'), 'Mehr Effektiv C++ Programmieren',
'35 neue Wege zur Verbesserung Ihrer Entwuerfe und Programme');

insert into t_authors (lastname, firstname)
values ('Schlossnagle', 'George');

insert into t_books (authorid, title, subtitle)
values ('1'), 'Advanced PHP Programming', 'A practical guide');


The problem is the authorid of t_books: which value should I take for
authorid of table t_books.
If I took '1', the referenced value of the authorid from t_authors was not
taken from the authorid from t_books, but always the value '1'?
If I took '0' or others I get errors.
In phpmyamin I set the reference to t_authors.authorid in the
t_books.authorid field.

Best regards and many thanks

Andreas



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=awfief@gmail.com



--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Willy Mularto
*nix forums beginner


Joined: 31 Mar 2005
Posts: 5

PostPosted: Wed Jul 12, 2006 2:50 am    Post subject: Re: datetime issue on MySQL 4.x (SOLVED) Reply with quote

Hello,
I have solved this problem, thanks.

Regards


Willy
----- Original Message -----
From: Willy
To: mysql@lists.mysql.com
Sent: Wednesday, July 12, 2006 8:55 AM
Subject: datetime issue on MySQL 4.x


Hello,
I have a MyISAM table:

CREATE TABLE `dlr` (
`msisdn` varchar(20) NOT NULL default '',
`source` varchar(20) NOT NULL default '',
`operator_id` varchar(20) NOT NULL default '',
`sms_message` longtext NOT NULL,
`smsc_id` varchar(20) NOT NULL default '',
`sms_id` varchar(250) NOT NULL default '',
`dlr_smsc` text NOT NULL,
`dlr` tinyint(4) NOT NULL default '0',
`date_time` datetime NOT NULL default '0000-00-00 00:00:00'
) TYPE=MyISAM

I have a case here, on July 7 I imported some data into the table with this query:

INSERT INTO dlr (source, msisdn, operator_id, sms_message, smsc_id, sms_id, dlr_smsc, dlr, date_time)
VALUES
('366508', '1234', 4, '', 'IM33665', MD5(RAND()), 'RE:', '2', NOW()),
('366508', '1234', 4, 'id:0943381383 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:xxxx, 'IM33665', MD5(RAND()), 'RE:id:0943381383 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:xxxx', '2', '2006-06-28 03:17:00'),
('366508', '1234', 4, 'id:1226442088 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:xxxx', 'IM33665', MD5(RAND()), 'RE:id:1226442088 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:Silahkan tawar: XDA ', '2', '2006-06-28 03:17:00'),
('366508', '1234', 4, 'id:1861980498 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:xxxx', 'IM33665', MD5(RAND()), 'RE:id:1861980498 sub:001 dlvrd:000 submit date:0606280317 done date:0606290317 stat:EXPIRED err:999 text:xxxx', '2', '2006-06-28 03:17:00'),
('366508', '1234', 4, '', 'IM33665', MD5(RAND()), 'RE:', '2', NOW()).......

And when I try to fetch the data using PHP with this query:

SELECT operator_id, DATE_FORMAT(date_time,'%d-%m-%Y') AS tanggal, COUNT(*) AS TOTAL FROM dlr WHERE 1=1 AND dlr = 8 AND MONTH(date_time) = MONTH(NOW()) GROUP BY tanggal, operator_id ORDER BY date_time

The weird thing happened. Why does all the rows are shown or treated as July 7th data? If we see the imported data, there should be June 28, June 29.....until July 7. Please help and many thanks for any reply.

Regards


Willy
Back to top
Jacob, Raymond A Jr
*nix forums beginner


Joined: 21 Mar 2005
Posts: 27

PostPosted: Mon Jul 10, 2006 7:53 pm    Post subject: Re: How does one speed up delete-Again Reply with quote

It appears that every time I start query the event or the data table
gets Locked.
Could this have any affect on why it takes so long to delete records.

Grasping at straws,
Thank you,
Raymond




mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
1555 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305568',
'12', '2006-07-10 11:48:22. |
| 542 | root | localhost |snort | Query
| 1555 | Sending data | DELET
E


data
FROM data, event
WHERE data.cid = event.cid
AND event.timestamp < "2006-04-01"
|
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 1555 | | NULL

|
| 544 | root | localhost |snort | Query
| 0 | NULL | show
processlist
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--processlist
|
| 545 | root | localhost |snort | Query
| 308 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.00 sec)

mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
333 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 333 | | NULL

|
| 544 | root | localhost |snort | Query
| 0 | NULL | show
processlist
|
| 545 | root | localhost |snort | Query
| 333 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.00 sec)

mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
337 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 337 | | NULL

|
| 544 | root | localhost |snort | Query
| 0 | NULL | show
processlist
|
| 545 | root | localhost |snort | Query
| 337 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.01 sec)

.....

mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+---------------------------------------------------------
----------------------------------------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
1627 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 1627 | | NULL

|
| 545 | root | localhost |snort | Query
| 1627 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
| 546 | root | localhost | NULL |
Query | 0 | NULL | show
processlist
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.00 sec)
.....
mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
3955 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 3955 | | NULL

|
| 545 | root | localhost |snort | Query
| 3955 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
| 546 | root | localhost | NULL |
Query | 0 | NULL | show
processlist
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.00 sec)

mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
3958 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 3958 | | NULL

|
| 545 | root | localhost |snort | Query
| 3958 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
| 546 | root | localhost | NULL |
Query | 0 | NULL | show
processlist
|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
4 rows in set (0.00 sec)

mysql> show processlist;
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| Id | User | Host | db |
Command | Time | State | Info

|
+-----+-------+-----------------------------------+---------------+-----
----+------+--------------+------
------------------------------------------------------------------------
------------------------+
| 1 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 |snort | Query |
3960 | Locked | INSER
T INTO event (sid,cid,signature,timestamp) VALUES ('2', '13305782',
'12', '2006-07-10 12:16:30. |
| 543 |goat| xxxxxxxxxxxxxxxxxxxxxxxxxxx:62680 | snort_archive | Sleep
| 3960 | | NULL

|
| 545 | root | localhost | snort |
Query | 3960 | Sending data | DELET
E data FROM data, event
WHERE data.cid = event.cid AND event.timestamp < '2006-03-03'
|
| 546 | root | localhost | NULL |
Query | 0 | NULL | show
processlist
|
+-----+-------+-----------------------------------+---------------+-----
----+------+-

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Jacob, Raymond A Jr
*nix forums beginner


Joined: 21 Mar 2005
Posts: 27

PostPosted: Mon Jul 10, 2006 4:43 pm    Post subject: Re: How does one speed up delete-Again Reply with quote

Could the problem the Locked data table in this case?
mysql> show processlist;
+-----+-------+-----------------------------------+-------+---------+---
-----+--------------+---------------------------------------------------
---------------------------------------------------+
| Id | User | Host | db | Command |
Time | State | Info
|
+-----+-------+-----------------------------------+-------+---------+---
-----+--------------+---------------------------------------------------
---------------------------------------------------+
| 1 | goat | xxxxxxxxxxxxxxxxxxxxxxxxxxx:62693 | snort | Query |
424668 | Locked | INSERT INTO data (sid,cid,data_payload) VALUES
('2','13305243','00000000450000C10FEF40002F11D162CD9E |
| 524 | root | localhost | snort | Query |
424669 | Sending data | DELETE data from data JOIN sidtemp ON data.cid sidtemp.cid |
| 537 | root | localhost | snort | Query |
242922 | Locked | DELETE data
FROM data, event
WHERE data.cid = event.cid
AND event.timestamp < "2006-05-01" |
| 542 | root | localhost | NULL | Query |
0 | NULL | show processlist
|
+-----+-------+-----------------------------------+-------+---------+---
-----+--------------+-----------------------------

I killed 524 & 537, started over and CPU went down. The query is still
slow.
Do I need to bring the database down to single user mode(I don't know
the correct
Term so I just am guessing single user mode exists) then do the delete?

Thank you,
Raymond
Back to top
Brent Baisley
*nix forums Guru Wannabe


Joined: 10 Feb 2005
Posts: 122

PostPosted: Mon Jul 10, 2006 12:50 pm    Post subject: Re: How does one speed up delete-Again Reply with quote

I've had to delete large amounts of records from tables with over 200 millions records. You are correct in that you want to do it in
chunks. How you divide the deletes is up to you to decide. You would want to do it based on a indexed column, like a date, and
specify a range.
DELETE data
FROM data, event
WHERE data.cid=event.cid
AND event.timestamp between "2006-01-01" and "2006-01-07"

DELETE data
FROM data, event
WHERE data.cid=event.cid
AND event.timestamp between "2006-01-08" and "2006-01-15"
....

You'l find it will go much, much quicker if you can narrow down the number of records deleted at one time. A delete that I would
give up on after a few hours take a few minutes by doing it in chunks.

BUT, if your data is always going to be divided into date chunks where you'll periodically delete everything prior to a certain
date, you may want to look into merge tables. Typically a merge table is a psuedo table that is comprised of a set of underlying
tables. You can add/drop underlying tables very quickly, or you could even reference an underlying table directly for certain tasks.
Under the right circumstances, merge tables can provide considerable amount of scalability.

----- Original Message -----
From: "Jacob, Raymond A Jr" <raymond.jacob@navy.mil>
To: <mysql@lists.mysql.com>
Sent: Sunday, July 09, 2006 10:36 PM
Subject: How does one speed up delete-Again


I started the operation below on Friday at 1300hrs EST
DELETE data
FROM data, event
WHERE data.cid = event.cid
AND event.timestamp < "2006-05-01"


It is now Sunday 22:00hrs EST and the operation is still running.

Question: Should it take this long to delete 7.5 million records from a
4.5GB
Table?

Question: Other than writing a script to export all the cid's to a file
and deleting the records one at a time so at least I can delete some
records.
Is there a way to delete records one at a time or in groups
so that if I have to stop the operation the delete will not rolled back?

Question:Does anyone on the list have experience deleting what I guess
is a large number of
records from a large table? i.e. how long does it take?

r/Raymond


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=brenttech@gmail.com


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql?unsub=gcdmg-mysql@m.gmane.org
Back to top
Google

Back to top
Display posts from previous:   
Post new topic   Reply to topic Page 1 of 67 [996 Posts] Goto page:  1, 2, 3, ..., 65, 66, 67 Next
View previous topic :: View next topic
The time now is Tue Dec 02, 2008 5:55 am | All times are GMT
navigation Forum index » Databases » MySQL
Jump to:  

Similar Topics
Topic Author Forum Replies Last Post
No new posts Success delivery report kennykilla Postfix 0 Wed Aug 29, 2007 9:06 am
No new posts Bug Report - Solaris 10 spell Richard B. Gilbert Solaris 0 Fri Jul 21, 2006 11:22 am
No new posts Work-needing packages report for Jul 21, 2006 wnpp@debian.org devel 0 Fri Jul 21, 2006 6:50 am
No new posts Work-needing packages report for Jul 14, 2006 wnpp@debian.org devel 0 Fri Jul 14, 2006 6:50 am
No new posts Work-needing packages report for Jul 7, 2006 wnpp@debian.org devel 0 Fri Jul 07, 2006 6:50 am

Xecuter 3 Mod Chip | Buy PSP | Mortgages | Mortgages | Car Finance
Copyright © 2004-2005 DeniX Solutions SRL
 
Other DeniX Solutions sites: Unix/Linux blog |  electronics forum |  medicine forum |  science forum | 
Privacy Policy


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.4951s ][ Queries: 16 (0.2967s) ][ GZIP on - Debug on ]